2021/06/22 by Hadi Mansourifar, Mansourifar, Hadi, Dana Alsagheer +7
Computer Science · #Advanced Malware Detection Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Internet Traffic Analysis and Secure E-voting
paper · pdf · doi:10.48550/arxiv.2107.02024
openalex publication_date 2021/06/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Hate speech detection has become a hot topic in recent years due to the exponential growth of offensive language in social media. It has proven that, state-of-the-art hate speech classifiers are efficient only when tested on the data with the same feature distribution as training data. As a consequence, model architecture plays the second role to improve the current results. In such a diverse data distribution relying on low level features is the main cause of deficiency due to natural bias in data. That's why we need to use high level features to avoid a biased judgement. In this paper, we statistically analyze the Perspective Scores and their impact on hate speech detection. We show that, different hate speech datasets are very similar when it comes to extract their Perspective Scores. Eventually, we prove that, over-sampling the Perspective Scores of a hate speech dataset can significantly improve the generalization performance when it comes to be tested on other hate speech datasets.