Extractive versus Generative Language Models for Political Conflict Text Classification

2025/12/31 by Patrick T. Brandt, Sultan Alsarra, Vito D’Orazio +5
Computer Science · Social Sciences · #Computational and Text Analysis Methods #Hate Speech and Cyberbullying Detection #Sentiment Analysis and Opinion Mining

paper · pdf · doi:10.1017/pan.2025.10027

openalex created_date 2025/12/31 · openalex publication_date 2025/12/31 · openalex updated_date 2026/07/02

Abstract

Abstract We review our recent ConfliBERT language model (Hu et al . 2022 [ConfliBERT: A Pre-Trained Language Model for Political Conflict and Violence]) to process political and violence-related texts. When fine-tuned, results show that ConfliBERT has superior performance in accuracy, precision, and recall over other large language models (LLMs) like Google’s Gemma 2 (9B), Meta’s Llama 3.1 (7B), and Alibaba’s Qwen 2.5 (14B) within its relevant domains. It is also hundreds of times faster than these more generalist LLMs. These results are illustrated using texts from the BBC, re3d, and the Global Terrorism Database. We demonstrate that open, fine-tuned models can outperform the more general models in terms of accuracy, precision, and recall, and at a fraction of the cost.

Citations

• Introducing the Global Terrorism Database
• Introducing the UCDP Georeferenced Event Dataset
• Green AI
• Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Text
• Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification
• Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems
• Rethinking the Event Coding Pipeline with Prompt Entailment
• Efficient Classification of Long Documents Using Transformers
• MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction
• Few-Shot Upsampling for Protest Size Detection
• LEGAL-BERT: The Muppets straight out of Law School
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• Green AI

Related

• ConfliBERT: A Language Model for Political Conflict
• Large language models as a substitute for human experts in annotating political text
• Large Language Models Outperform Expert Coders and Supervised Classifiers at Annotating Political Social Media Messages
• Positioning Political Texts with Large Language Models by Asking and Averaging
• Positioning Political Texts with Large Language Models by Asking and Averaging