A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labelling

2025/10/16 by Shiyu Ji, Ji, Shiyu, Farnoosh Hashemi +20
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Misinformation and Its Impacts #Sentiment Analysis and Opinion Mining #Social and Information Networks (cs.SI)

paper · pdf · doi:10.48550/arxiv.2510.15081

openalex publication_date 2025/10/16 · openalex created_date 2025/10/21 · openalex updated_date 2026/07/28

Abstract

Rhetorical strategies are central to persuasive communication, from political discourse and marketing to legal argumentation. However, analysis of rhetorical strategies has been limited by reliance on human annotation, which is costly, inconsistent, difficult to scale. Their associated datasets are often limited to specific topics and strategies, posing challenges for robust model development. We propose a novel framework that leverages large language models (LLMs) to automatically generate and label synthetic debate data based on a four-part rhetorical typology (causal, empirical, emotional, moral). We fine-tune transformer-based classifiers on this LLM-labeled dataset and validate its performance against human-labeled data on this dataset and on multiple external corpora. Our model achieves high performance and strong generalization across topical domains. We illustrate two applications with the fine-tuned model: (1) the improvement in persuasiveness prediction from incorporating rhetorical strategy labels, and (2) analyzing temporal and partisan shifts in rhetorical strategies in U.S. Presidential debates (1960-2020), revealing increased use of affective over cognitive argument in U.S. Presidential debates.

Citations

• Communication is All You Need: Persuasion Dataset Construction via Multi-LLM Communication
• Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
• Generating Educational Materials with Different Levels of Readability using LLMs
• Adapted large language models can outperform medical experts in clinical text summarization
• PatternGPT :A Pattern-Driven Framework for Large Language Model Text Generation
• Out of One, Many: Using Language Models to Simulate Human Samples
• Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
• Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
• Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests
• Persuasive Natural Language Generation -- A Literature Review
• Examining the Ordering of Rhetorical Strategies in Persuasive Requests
• An Unsupervised Domain-Independent Framework for Automated Detection of Persuasion Tactics in Text
• A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis
• Automatic Argument Quality Assessment -- New Datasets and Methods
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good
• Finding Convincing Arguments Using Scalable Bayesian Preference Learning
• Mass Polarization: Manifestations and Measurements

Related

• LLM-POTUS Score: A Framework of Analyzing Presidential Debates with Large Language Models
• Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns
• Analyzing Biases in Political Dialogue: Tagging U.S. Presidential Debates with an Extended DAMSL Framework
• Can Language Models Recognize Convincing Arguments?
• Analysing Personal Attacks in U.S. Presidential Debates