2023/07/12 by Nicholas Perikli, Perikli, Nicholas, Srimoy Bhattacharya +19 · 1 citation
Computer Science · Medicine · Social Sciences · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Computers and Society (cs.CY) #Coronavirus disease 2019 (COVID-19) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Hyperparameter #Machine Learning (cs.LG) #Machine learning #Medicine #Misinformation and Its Impacts #Natural language processing #Reliability (semiconductor) #Sentiment analysis #Social and Information Networks (cs.SI) #Social media #Support vector machine #Vaccine Coverage and Hesitancy #World Wide Web
paper · pdf · doi:10.48550/arxiv.2307.15072
openalex publication_date 2023/07/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Very few social media studies have been done on South African user-generated content during the COVID-19 pandemic and even fewer using hand-labelling over automated methods. Vaccination is a major tool in the fight against the pandemic, but vaccine hesitancy jeopardizes any public health effort. In this study, sentiment analysis on South African tweets related to vaccine hesitancy was performed, with the aim of training AI-mediated classification models and assessing their reliability in categorizing UGC. A dataset of 30000 tweets from South Africa were extracted and hand-labelled into one of three sentiment classes: positive, negative, neutral. The machine learning models used were LSTM, bi-LSTM, SVM, BERT-base-cased and the RoBERTa-base models, whereby their hyperparameters were carefully chosen and tuned using the WandB platform. We used two different approaches when we pre-processed our data for comparison: one was semantics-based, while the other was corpus-based. The pre-processing of the tweets in our dataset was performed using both methods, respectively. All models were found to have low F1-scores within a range of 45%-55%, except for BERT and RoBERTa which both achieved significantly better measures with overall F1-scores of 60% and 61%, respectively. Topic modelling using an LDA was performed on the miss-classified tweets of the RoBERTa model to gain insight on how to further improve model accuracy.