CovRelex-SE: Adding Semantic Information for Relation Search via Sequence Embedding

In recent years, COVID-19 has impacted all aspects of human life. As a result, numerous publications relating to this disease have been issued. Due to the massive volume of publications, some retrieval systems have been developed to provide researchers with useful information. In these systems, lexical searching methods are widely used, which raises many issues related to acronyms, synonyms, and rare keywrds. In this paper, we present a hybrid relation retrieval system, CovRelex-SE, based on embeddings to provide high-quality search results. Our system can be accessed through the following URL: https://www.jaist.ac.jp/is/labs/nguyen-lab/systems/covrelex-se/. © 2023 Association for Computational Linguistics.

Detecting COVID-19 Vaccine Stance and Symptom Reporting from Tweets using Contextual Embeddings

Determining sentiments of the public with regard to COVID-19 vaccines is crucial for nations to efficiently carry out vaccination drives and spread awareness. Hence, it is a field requiring accurate analysis and captures the interest of many researchers. Microblogs from social media websites such as Twitter sometimes contain colloquial expressions or terminology difficult to interpret making the task a challenging one. In this paper, we propose a method for multi-label text classification for the track of”Information Retrieval from Microblogs during Disasters (IRMiDis)” presented by the”Forum of Information Retrieval Evaluation” in 2022, related to vaccine sentiment among the public and reporting of someone experiencing COVID-19 symptoms. The following methodologies have been utilised: (i) Word2Vec and (ii) BERT, which uses contextual embedding rather than the fixed embedding used by conventional natural language models. For Task 1, the overall F1 score and Accuracy are 0.503 and 0.529, respectively, placing us fourth among all the teams, while for Task 2, they are 0.740 and 0.790, placing us second among all the teams who submitted their work. Our code is openly accessible through GitHub. 1 © 2022 Copyright for this paper by its authors.

Mining Trends of COVID-19 Vaccine Beliefs on Twitter With Lexical Embeddings: Longitudinal Observational Study.

Background: Social media plays a pivotal role in disseminating news globally and acts as a platform for people to express their opinions on various topics. A wide variety of views accompany COVID-19 vaccination drives across the globe, often colored by emotions that change along with rising cases, approval of vaccines, and multiple factors discussed online. Objective: This study aims to analyze the temporal evolution of different emotions and the related influencing factors in tweets belonging to 5 countries with vital vaccine rollout programs, namely India, the United States, Brazil, the United Kingdom, and Australia. Methods: We extracted a corpus of nearly 1.8 million Twitter posts related to COVID-19 vaccination and created 2 classes of lexical categories-emotions and influencing factors. Using cosine distance from selected seed words' embeddings, we expanded the vocabulary of each category and tracked the longitudinal change in their strength from June 2020 to April 2021 in each country. Community detection algorithms were used to find modules in positive correlation networks. Results: Our findings indicated the varying relationship among emotions and influencing factors across countries. Tweets expressing hesitancy toward vaccines represented the highest mentions of health-related effects in all countries, which reduced from 41% to 39% in India. We also observed a significant change (P<.001) in the linear trends of categories like hesitation and contentment before and after approval of vaccines. After the vaccine approval, 42% of tweets coming from India and 45% of tweets from the United States represented the "vaccine_rollout" category. Negative emotions like rage and sorrow gained the highest importance in the alluvial diagram and formed a significant module with all the influencing factors in April 2021, when India observed the second wave of COVID-19 cases. Conclusions: By extracting and visualizing these tweets, we propose that such a framework may help guide the design of effective vaccine campaigns and be used by policy makers to model vaccine uptake and targeted interventions.

ZnO coated nonwoven polypropylene fabric with antibacterial and antiviral function for medical applications

The present pandemic has highlighted the necessity of infection protection gear as a crucial protective approach, particularly given the fact that severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) readily infects individuals in poorly ventilated environments. Embedding antimicrobial function onto protection gear would have major implications in minimizing pathogen contamination and lowering healthcare associated illness. In this study, non woven polypropylene fabric (NWPP) which is widely used in personal hygiene products and hospital protective gears has been subjected to surface fictionalization with corona treatment. Surface polarity of the treated fabric was studied by use of dyne liquid which showed generation of surface polarization. Subsequently, the resultant surface polarized NWPP were spray coated with zinc oxide (ZnO) antiviral agent. The antiviral agents were rendered to adhere to NWPP by use of polyurethane solution coating on the fabric. The effect of antiviral coatings on NWPP fabric with the use of polyurethane solution as an adhesive were investigated in terms of antiviral activity and anti-bacterial activity against MS2 bacteriophage and Staphylococcus aureus and Klebsiella pneumonia bacteria respectively. Coating of surface polarized NWPP with polyurethene binder reduced the leaching of antiviral coating. More importantly, the fabrics exhibited promising antiviral and anti bacterial activity with 99.90 % reduction in microorganisms after 24 hours of exposure. © 2022 IEEE.

Deep Metric Learning for Transparent Classification of Covid-19 X-Ray Images

This work proposes an interpretable classifier for automatic Covid-19 classification using chest X-ray images. It is based on a deep learning model, in particular, a triplet network, devoted to finding an effective image embedding. Such embedding is a non-linear projection of the images into a space of reduced dimension, where homogeneity and separation of the classes measured by a predefined metric are improved. A K-Nearest Neighbor classifier is the interpretable model used for the final classification. Results on public datasets show that the proposed methodology can reach comparable results with state of the art in terms of accuracy, with the advantage of providing interpretability to the classification, a characteristic which can be very useful in the medical domain, e.g. in a decision support system. © 2022 IEEE.

Designing for the Embedding of Employee Voice

Previous research on employee voice has sought to design technological solutions that address the challenges of speaking up in the workplace. However, effectively embedding employee voice systems in organisations requires designers to engage with the social processes, power relations and contextual factors of individual workplaces. We explore this process within a university workplace through a research project responding to a crisis in educational service delivery arising from the COVID-19 pandemic. Within a successful three-month staff-led engagement, we examined the intricacies of embedding employee voice, exploring how the interactions between existing actors impacted the effectiveness of the process. We sought to identify specific actions to promote employee voice and overcome barriers to its successful establishment in organisational decision-making. We highlight design considerations for an effective employee voice system that facilitates embedding employee voice, including assurance, bounded accountability and bias reflexivity. © 2023 ACM.

The Investigation into Deep Learning Classifiers Towards Imbalanced Text Data

Class imbalance is an important classification problem where failure to identify events can be hazardous due to failure of solution preparation or opportune handling. Minorities are mostly more consequential in such cases. It is necessary to know a reliable classifier for imbalanced classes. This study examines several conventional machine learning and deep learning methods to compare the performance of each method on dataset with imbalanced classes. We use COVID-19 online news titles to simulate different class imbalance ratios. The results of our study demonstrate the superiority of the CNN with embedding layer method on a news titles dataset of 16,844 data points towards imbalance ratios of 37%, 30%, 20%, 10%, and 1%. However, CNN with embedding layer showed a noticeable performance degradation at an imbalance ratio of 1%. © 2022 IEEE.

PRISTINE: Semi-supervised Deep Learning Opioid Crisis Detection on Reddit

The drug abuse epidemic has been on the rise in the past few years, particularly after the start of COVID-19 pandemic. Our preliminary observations on Reddit alone show that discussions on drugs from 2018 to 2020 increased between a range of 45% to 200%, and so has the number of unique users participating in those discussions. Existing efforts focused on utilizing social media to distinguish potential drug abuse chats from unharmful chats regardless of what drug is being abused. Others focused on understanding the trends and causes of drug abuse from social media. To this end, we introduce PRISTINE (opioid crisis detection on reddit), our work dynamically detects-and extracts evolving misleading drug names from Reddit comments using reinforced Dynamic Query Expansion (DQE) and constructs a textual Graph Convolutional Network with the aid of powerful pre-trained embeddings to detect which type of drug class a Reddit comment corresponds to. Further, we perform extensive experiments to investigate the effectiveness of our model. © 2022 IEEE.

Synonym-based Text Generation in Restructuring Imbalanced Dataset for Deep Learning Models

One of which machine learning data processing problems is imbalanced classes. Imbalanced classes could potentially cause bias towards the majority classes due to the nature of machine learning algorithms that presume that the object cardinality in classes is around similar number. Oversampling or generating new objects in minority class are common approaches for balancing the dataset. In text oversampling method, semantic meaning loses often occur when deep learning algorithms are used. We propose synonym-based text generation for restructuring the imbalanced COVID-19 online-news dataset. Three deep learning models (MLP, CNN, and LSTM) using TF/IDF and word embedding (WE) feature are tested with the original and balanced dataset. The results indicate that the balance condition of the dataset and the use of text representative features affect the performance of the deep learning model. Using balanced data and deep learning models with WE greatly affect the classification significantly higher performances as high as 4%, 5%, and 6% in accuracy, precision, recall, and f1-score, respectively. © 2022 IEEE.

Learning the Protein Language Model of SARS-CoV-2 Spike Proteins

Ahstract-SARS-CoV-2 virus has long been evolving posing an increased risk in terms of infectivity and transmissibility which causes greater impact in communities worldwide. With the surge of collected SARS-CoV-2 sequences, studies found out that most of the emerging variants are linked to increased mutations in the spike (S) protein as observed in Alpha, Beta, Gamma, and Delta variants. Multiple approaches on genomic surveillance have been performed to monitor the mutational status and spread of the virus however most are heavily dependent on labels attributed to these sequences. Hence, this study features a system that has the capability to learn the protein language model of SARS-CoV-2 spike proteins, based on a bidirectional long-short term memory (BiLSTM) recurrent neural network, using sequence data alone. Upon obtaining the sequence embedding from the model, observed clusters are generated using the Leiden clustering algorithm and is visualized to monitor similarities between variants in terms of grammatical probability and semantic change. Additionally, the system measures the validity of a user-generated next-generation sequence capturing potential sequence mutations indicative of viral escape, particularly mutations by substitutions. Further studies on methods uncovering semantic rules that govern spike proteins are recommended to learn more about other viral characteristics conclusive of the future of the COVID-19 pandemic. © 2023 IEEE.

Detect misinformation of COVID-19 using deep learning: A comparative study based on word embedding

Since its emergence in December 2019, there have been numerous news of COVID-19 pandemic shared on social media, which contain information from both reliable and unreliable medical sources. News and misleading information spread quickly on social media, which can lead to anxiety, unwanted exposure to medical remedies, etc. Rapid detection of fake news can reduce their spread. In this paper, we aim to create an intelligent system to detect misleading information about COVID-19 using deep learning techniques based on LSTM and BLSTM architectures. Data used to construct the DL models are text type and need to be transformed to numbers. We test, in this paper the efficiency of three vectorization techniques: Bag of words, Word2Vec and Bert. The experimental study showed that the best performance was given by LSTM model with BERT by achieving an accuracy of 91% of the test set. © 2023 IEEE.

HateNet: A Graph Convolutional Network Approach to Hate Speech Detection

The COVID-19 pandemic has caused hate speech on online social networks to become a growing issue in recent years, affecting millions. Our work aims to improve automatic hate speech detection to prevent escalation to hate crimes. The first c hallenge i n h ate s peech r esearch i s t hat e xisting datasets suffer from quite severe class imbalances. The second challenge is the sparsity of information in textual data. The third challenge is the difficulty i n b alancing t he t radeoff b etween utilizing semantic similarity and noisy network language. To combat these challenges, we establish a framework for automatic short text data augmentation by using a semi-supervised hybrid of Substitution Based Augmentation and Dynamic Query Expansion (DQE), which we refer to as SubDQE, to extract more data points from a specific c lass f rom T witter. W e a lso p ropose the HateNet model, which has two main components, a Graph Convolutional Network and a Weighted Drop-Edge. First, we propose a Graph Convolutional Network (GCN) classifier, using a graph constructed from the thresholded cosine similarities between tweet embeddings to provide new insights into how ideas are connected. Second, we propose a weighted Drop-Edge based stochastic regularization technique, which removes edges randomly based on weighted probabilities assigned by the semantic similarities between Tweets. Using 3 different SubDQE-augmented datasets, we compare our HateNet model using eight different tweet embedding methods, six other baseline classification models, and seven other baseline data augmentation techniques previously used in the realm of hate speech detection. Our results show that our proposed HateNet model matches or exceeds the performance of the baseline models, as indicated by the accuracy and F1 score. © 2022 IEEE.

Ensemble learning with speaker embeddings in multiple speech task stimuli for depression detection.

Introduction: As a biomarker of depression, speech signal has attracted the interest of many researchers due to its characteristics of easy collection and non-invasive. However, subjects' speech variation under different scenes and emotional stimuli, the insufficient amount of depression speech data for deep learning, and the variable length of speech frame-level features have an impact on the recognition performance. Methods: The above problems, this study proposes a multi-task ensemble learning method based on speaker embeddings for depression classification. First, we extract the Mel Frequency Cepstral Coefficients (MFCC), the Perceptual Linear Predictive Coefficients (PLP), and the Filter Bank (FBANK) from the out-domain dataset (CN-Celeb) and train the Resnet x-vector extractor, Time delay neural network (TDNN) x-vector extractor, and i-vector extractor. Then, we extract the corresponding speaker embeddings of fixed length from the depression speech database of the Gansu Provincial Key Laboratory of Wearable Computing. Support Vector Machine (SVM) and Random Forest (RF) are used to obtain the classification results of speaker embeddings in nine speech tasks. To make full use of the information of speech tasks with different scenes and emotions, we aggregate the classification results of nine tasks into new features and then obtain the final classification results by using Multilayer Perceptron (MLP). In order to take advantage of the complementary effects of different features, Resnet x-vectors based on different acoustic features are fused in the ensemble learning method. Results: Experimental results demonstrate that (1) MFCC-based Resnet x-vectors perform best among the nine speaker embeddings for depression detection; (2) interview speech is better than picture descriptions speech, and neutral stimulus is the best among the three emotional valences in the depression recognition task; (3) our multi-task ensemble learning method with MFCC-based Resnet x-vectors can effectively identify depressed patients; (4) in all cases, the combination of MFCC-based Resnet x-vectors and PLP-based Resnet x-vectors in our ensemble learning method achieves the best results, outperforming other literature studies using the depression speech database. Discussion: Our multi-task ensemble learning method with MFCC-based Resnet x-vectors can fuse the depression related information of different stimuli effectively, which provides a new approach for depression detection. The limitation of this method is that speaker embeddings extractors were pre-trained on the out-domain dataset. We will consider using the augmented in-domain dataset for pre-training to improve the depression recognition performance further.

Estimating the effect of COVID-19 on mental health: Linguistic indicators of depression during a global pandemic

This preliminary analysis uses a deep LSTM neural network with fastText embeddings to predict population rates of depression on Reddit in order to estimate the effect of COVID-19 on mental health. We find that year over year, depression rates on Reddit are up 50%, suggesting a 15-million person increase in the number of depressed Americans and a $7.5 billion increase in depression related spending. This finding comes at a time when uncertainty about the impact of COVID-19 on physical and economic health is still high, and suggests that in addition to those factors, mental health must be considered as well. As data becomes available, further research will be needed to validate the results of this preliminary investigation. © ACL 2020.All right reserved.

Transformers for COVID-19 Misinformation Detection on Twitter: A South African Case Study

The aim of this paper is to investigate the use of transformer-based neural network classifiers for the detection of misinformation on South African Twitter. Twitter COVID-19 misinformation data from four publicly available datasets are used for training. Four different transformer-based embedding methods are used, namely: BERT, CT-BERT, ELECTRA, and LAMBERT. A neural network classifier is trained for each embedding method, and the architectures are optimized with the hyperband optimization algorithm. The model using the LAMBERT embedding method attains the highest F1-score (0.899) on the test data. The model does not generalize well to the South African context however, since it fails to reliably distinguish between general Tweets and COVID-19 misinformation Tweets when applied to the unlabeled South African data. The classifier does detect instances of misinformation that are consistent with known COVID-19 misinformation spread in South Africa, but these are in the minority. It is therefore recommended that misinformation datasets specific to the South African context be curated in order to facilitate future research efforts dedicated to misinformation detection on South African Twitter. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Comparison Analysis of LSTM and CNN Variants with Embedding Word Methods for Sentiment Analysis on Food Consumption Behavior

Lockdowns, working from home, staying at home, and physical distance are expected to significantly impact consumer attitudes and behaviors during the COVID-19 pandemic. During the implementation of the Movement Control Order, Malaysians' food preferences are already shifting away, influencing new consumption behavior. Since it has played a significant role in many areas of natural language, mainly using social media data from Twitter, there has been increased interest in sentiment analysis in recent years. However, research on the performance of various sentiment analysis methodologies such as n-gram ranges, lexicon techniques, deep learning, word embedding, and hybrid methods within this domain-specific sentiment is limited. This study evaluates several approaches to determine the best approach for tweets on food consumption behavior in Malaysia during the COVID-19 pandemic. This study combined unigram and bigram ranges with two lexicon-based techniques, TextBlob and VADER, and three deep learning classi-fiers, Long Short-Term Memory Network (LSTM), Convolutional Neural Networks (CNN), and their hybridization. Word2Vector and GloVe are two-word embedding approaches used by LSTM-CNN. The embedding GloVe on TextBlob approach with a combination of Unigram + Bigram [1,2] range produced the best results, with 85.79% accuracy and 85.30% F1-score. According to these findings, LSTM outperforms other classifiers because it achieves the highest scores for both performance metrics. The classification performance can be improved in future studies if the dataset is more evenly distributed across each positive and negative label. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

NLP Text Classification for COVID-19 Automatic Detection from Radiology Report in Indonesian Language

Radiology is used as an important assessment for patients with pulmonary disease. The radiology images are usually accompanied by a written report from a radiologist to be passed to the other referring physicians. These radiology reports are written in a natural language where they can have different systematic structures based on the language used. In our study, the radiology reports were collected from an Indonesian hospital and written in Bahasa Indonesia. We performed an automatic text classification to differentiate the information written in the radiology reports into two classes, COVID-19 and non COVID-19. To find the best model, we evaluated several embedding techniques available for Bahasa and five Machine Learning (ML) models, namely (1) XGBoost, (2) fastText, (3) LSTM, (4) Bi-LSTM and (5) IndoBERT. The result shows that IndoBERT outperformed the others with an accuracy of 98%. In terms of training speed, the shallow neural network architecture implemented with the fastText library can train the model in under one second and still result in a reasonably good accuracy of 86%. © 2022 IEEE.

An End-to-End Knowledge Graph Based Question Answering Approach for COVID-19

Question Answering based on Knowledge Graph (KG) has emerged as a popular research area in general domain. However, few works focus on the COVID-19 kg-based question answering, which is very valuable for biomedical domain. In addition, existing question answering methods rely on knowledge embedding models to represent knowledge (i.e., entities and questions), but the relations between entities are neglected. In this paper, we construct a COVID-19 knowledge graph and propose an end-to-end knowledge graph question answering approach that can utilize relation information to improve the performance. Experimental result shows that the effectiveness of our approach on the COVID-19 knowledge graph question answering. Our code and data are available at https://github.com/CHNcreater/COVID-19-KGQA. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

LexDeep: Hybrid Lexicon and Deep Learning Sentiment Analysis Using Twitter for Unemployment-Related Discussions during COVID-19

The COVID-19 pandemic has spread globally, resulting in financial instability in many countries and reductions in the per capita gross domestic product. Sentiment analysis is a cost-effective method for acquiring sentiments based on household income loss, as expressed on social media. However, limited research has been conducted in this domain using the LexDeep approach. This study aimed to explore social trend analytics using LexDeep, which is a hybrid sentiment analysis technique, on Twitter to capture the risk of household income loss during the COVID-19 pandemic. First, tweet data were collected using Twint with relevant keywords before (9 March 2019 to 17 March 2020) and during (18 March 2020 to 21 August 2021) the pandemic. Subsequently, the tweets were annotated using VADER (lexicon-based) and fed into deep learning classifiers, and experiments were conducted using several embeddings, namely simple embedding, Global Vectors, and Word2Vec, to classify the sentiments expressed in the tweets. The performance of each LexDeep model was evaluated and compared with that of a support vector machine (SVM). Finally, the unemployment rates before and during COVID-19 were analysed to gain insights into the differences in unemployment percentages through social media input and analysis. The results demonstrated that all LexDeep models with simple embedding outperformed the SVM. This confirmed the superiority of the proposed LexDeep model over a classical machine learning classifier in performing sentiment analysis tasks for domain-specific sentiments. In terms of the risk of income loss, the unemployment issue is highly politicised on both the regional and global scales;thus, if a country cannot combat this issue, the global economy will also be affected. Future research should develop a utility maximisation algorithm for household welfare evaluation, given the percentage risk of income loss owing to COVID-19. © 2023 Tech Science Press. All rights reserved.

International Workshop on Learning with Knowledge Graphs: Construction, Embedding, and Reasoning

A knowledge graph (KG) consists of numerous triples, in which each triple, i.e., (head entity, relation, tail entity), denotes a real-world assertion. Many large-scale KGs have been developed, e.g., general-purpose KGs Freebase and YAGO. Also, lots of domain-specific KGs are emerging, e.g., COVID-19 KGs, biomedical KGs, and agricultural KGs. By embedding KGs into low-dimensional vectors, i.e., representations of entities and relations, we could integrate KGs into machine learning models and enhance the performance of many prediction tasks, including search, recommendations, and question answering. During the construction, refinement, embedding, and application of KGs, a variety of KG learning algorithms have been developed to handle challenges in various real-world scenarios. Moreover, graph neural networks have also brought new opportunities to KG learning. This workshop aims to engage with active researchers from KG communities, recommendation communities, natural language processing communities, and other communities, and deliver state-of-the-art research insights into the core challenges in KG learning. © 2023 Owner/Author.