TM-BERT: A Twitter Modified BERT for Sentiment Analysis on Covid-19 Vaccination Tweets
2nd International Conference on Digital Futures and Transformative Technologies, ICoDT2 2022
; 2022.
Article
in English
| Scopus | ID: covidwho-1922689
ABSTRACT
In transfer learning a model is pre-trained on a large unsupervised dataset and then fine-tuned on domain-specific downstream tasks. BERT is the first true-natured deep bidirectional language model which reads the input from both sides of input to better understand the context of a sentence by solely relying on the Attention mechanism. This study presents a Twitter Modified BERT (TM-BERT) based upon Transformer architecture. It has also developed a new Covid-19 Vaccination Sentiment Analysis Task (CV-SAT) and a COVID-19 unsupervised pre-training dataset containing (70K) tweets. BERT achieved (0.70) and (0.76) accuracy when fine-tuned on CV-SAT, whereas TM-BERT achieved (0.89), a (19%) and (13%) accuracy over BERT. Another enhancement introduced is in terms of time efficiency as BERT takes (64) hours of pre-training while TM-BERT takes only (17) hours and still produces (19%) improvement even after pre-trained on four (4) times fewer data. © 2022 IEEE.
Full text:
Available
Collection:
Databases of international organizations
Database:
Scopus
Topics:
Vaccines
Language:
English
Journal:
2nd International Conference on Digital Futures and Transformative Technologies, ICoDT2 2022
Year:
2022
Document Type:
Article
Similar
MEDLINE
...
LILACS
LIS