2022/05/02 by Yujian Liu, Liu, Yujian, Xinliang Frederick Zhang +7 · 8 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #cs.CL
paper · pdf · doi:10.48550/arxiv.2205.00619
Findings of NAACL'22. The first two authors contribute equally
arxiv created 2022/05/02 · openalex publication_date 2022/05/02 · arxiv updated 2022/05/03 · openalex created_date 2023/03/20 · openalex updated_date 2026/07/28
Ideology is at the core of political science research. Yet, there still does not exist general-purpose tools to characterize and predict ideology across different genres of text. To this end, we study Pretrained Language Models using novel ideology-driven pretraining objectives that rely on the comparison of articles on the same story written by media of different ideologies. We further collect a large-scale dataset, consisting of more than 3.6M political news articles, for pretraining. Our model POLITICS outperforms strong baselines and the previous state-of-the-art models on ideology prediction and stance detection tasks. Further analyses show that POLITICS is especially good at understanding long or formally written texts, and is also robust in few-shot learning scenarios.