A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

2026/06/03 by Jared Moore, Noah Goodman, Nick Haber +1 · 1 voice
Computer Science · #cs.CL #cs.AI #cs.HC

paper · pdf

Abstract

Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint measures identify whether persuasion occurred, yet miss where and how beliefs moved within a dialogue. We present PERSUASIONTRACE, a framework for studying persuasion in human-LLM interaction. Built on a web-based experimental platform, PERSUASIONTRACE contributes a tool for multi-turn persuasion studies and a process-level evaluation protocol: it records multi-turn belief reports from human or simulated targets of persuasion, annotates persuader turns with rhetorical dimensions (logos/pathos/ethos), and evaluates simulators by fidelity to real human belief dynamics. Using this framework, we find that human targets group into two clusters of multi-turn belief updates and exhibit susceptibility to rhetorical strategies, and that LLMs are persuasive across generic and personalized topics, text and audio modalities, and multi-turn interactions. Prior work has chiefly used vanilla-prompted LLMs to simulate human targets, but we show that these simulators fail to replicate human belief dynamics. We introduce a Bayesian-network simulated target that maintains an explicit latent belief state over time so each persuader message yields cognitively realistic belief updates. In human-likeness evaluation, our Bayesian target scores near a human reference (81 vs 80), while baseline LLM targets score substantially lower (64). PERSUASIONTRACE reframes persuasion evaluation from endpoint movement alone to process fidelity, providing a stronger basis for scientific analysis and safer optimization of persuasive systems.

Citations

• : Manipulation: Its Nature, Mechanisms, and Moral Status
• Characterizing Delusional Spirals through Human-LLM Chat Logs
• Training LLMs for Honesty via Confessions
• The Effect of Belief Boxes and Open-mindedness on Persuasion
• A Meta-Analysis of the Persuasive Power of Large Language Models
• A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models
• Difficulties with Evaluating a Deception Detector for AIs
• A Two-Step, Multidimensional Account of Deception in Language Models
• Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability
• Epistemic Diversity and Knowledge Collapse in Large Language Models
• D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models
• Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Task
• The Levers of Political Persuasion with Conversational AI
• PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation
• The Lock-in Hypothesis: Stagnation by Algorithm
• It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
• ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind
• Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
• How malicious AI swarms can threaten democracy
• LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models
• Scaling language model size yields diminishing returns for single-message political persuasion
• Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
• PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues
• Tailored Truths: Optimizing LLM Persuasion with Personalization and Fabricated Statistics
• Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications
• LLM-initialized Differentiable Causal Discovery
• AI can help humans find common ground in democratic deliberation
• On the Reliability of Large Language Models for Causal Discovery
• Are Large Language Models Consistent over Value-laden Questions?
• Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language
• Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Ranking
• A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
• Can Language Models Recognize Convincing Arguments?
• On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
• Evaluating Frontier Models for Dangerous Capabilities
• From "um" to "yeah": Producing, predicting, and regulating information flow in human conversation
• Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?
• Debating with More Persuasive LLMs Leads to More Truthful Answers
• How Experiments Help Campaigns Persuade Voters: Evidence from a Large Archive of Campaigns’ Own Experiments
• A Roadmap to Pluralistic Alignment
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• The Persuasive Power of Large Language Models
• AI Control: Improving Safety Despite Intentional Subversion
• Honesty Is the Best Policy: Defining and Mitigating AI Deception
• The Adoption and Efficacy of Large Language Models: Evidence From Consumer Complaints in the Financial Industry
• Leveraging AI for democratic discourse: Chat interventions can improve online political conversations at scale
• AI Deception: A Survey of Examples, Risks, and Potential Solutions
• S³: Social-network Simulation System with Large Language Model-Empowered Agents
• Towards Measuring the Representation of Subjective Global Opinions in Language Models
• From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
• Model evaluation for extreme risks
• Characterizing Manipulation from AI Systems
• Training language models to follow instructions with human feedback
• Influence via Ethos: On the Persuasive Power of Reputation in Deliberation Online
• Changing Views: Persuasion Modeling and Argument Extraction from Online Discussions
• A Model of Competing Narratives
• AI safety via debate
• Attentive Interaction Model: Modeling Changes in View in Argumentation
• Argument Strength is in the Eye of the Beholder: Audience Effects in Persuasion
• Concrete Problems in AI Safety
• Why do humans reason? Arguments for an argumentative theory
• Social Influence: Compliance and Conformity
• Asking About Attitude Change
• Epistemic Vigilance
• The three faces of Eve: Strategic displays of positive, negative, and neutral emotions in negotiations

Discussions

• Bottom line: endpoint movement alone is not enough. Process fidelity matters: when beliefs move, how they move, and whether simulator updates look human. Code: github.com/jlcmoore/per... Preprint: arx [bsky, 3 points, 0 comments]

Related

• How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations
• Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
• Persuasiveness and Bias in LLM: Investigating the Impact of Persuasiveness and Reinforcement of Bias in Language Models
• A Hybrid Theory and Data-driven Approach to Persuasion Detection with Large Language Models
• Tailored Truths: Optimizing LLM Persuasion with Personalization and Fabricated Statistics