Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications

2024/11/11 by Sander Noels, Rogiers, Alexander, Alexander Rogiers +5 · 1 voice · 9 citations
Arts and Humanities · Physics and Astronomy · Social Sciences · #Media Influence and Health #Misinformation and Its Impacts #Opinion Dynamics and Social Influence #cs.CL

paper · pdf · doi:10.48550/arxiv.2411.06837

openalex publication_date 2024/11/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The rapid rise of Large Language Models (LLMs) has created new disruptive possibilities for persuasive communication, enabling fully-automated, personalized, and interactive content generation at an unprecedented scale. In this paper, we survey the emerging field of LLM-based persuasion, reviewing empirical studies that measure the influence of LLM Systems on human attitudes and behaviors. We categorize applications across domains such as politics, marketing, public health, e-commerce, and charitable giving, finding that such systems have frequently achieved human-level or even superhuman persuasiveness. Synthesizing recent evidence, we identify key factors influencing this effectiveness, including the interaction approach, model scale and capability, prompt design, personalization, and AI source disclosure. Furthermore, we critically examine the experimental designs and success metrics used to evaluate these Systems, distinguishing between direct behavioral outcomes and proxy indicators. Our survey suggests that the current capabilities of LLM-based persuasion pose profound ethical and societal risks, including to information integrity, fairness and inclusion, privacy, and individual autonomy. These risks underscore the urgent need for ethical guidelines and updated regulatory frameworks to avoid the widespread deployment of irresponsible and harmful LLM Systems.

Cited by

• Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
• Biased AI writing assistants shift users’ attitudes on societal issues
• Perceived Political Bias in LLMs Reduces Persuasive Abilities
• Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions
• Structuring Collective Action with LLM-Guided Evolution: From Ill-Structured Problems to Executable Heuristics
• A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models
• A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing
• How ChatGPT and Gemini View the Elements of Communication Competence of Large Language Models: A Pilot Study
• Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD
• How Persuasive Could LLMs Be? A First Study Combining Linguistic-Rhetorical Analysis and User Experiments

Discussions

• This recent survey has more examples of AI persuasive powers -- but wait, there's an executive order! "we must develop AI systems that are free from ideological bias or engineered social agendas!" Tha [bsky, 5 points, 1 comments]

Related

• Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
• Measuring and Improving Persuasiveness of Large Language Models
• On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
• Persuasion Games using Large Language Models
• Persuasion and Safety in the Era of Generative AI