When Large Language Models are More PersuasiveThan Incentivized Humans, and Why

2025/05/14 by Philipp Schoenegger, Schoenegger, Philipp, Francesco Salvi +77 · 31 voices · 8 citations
#cs.CL

paper · pdf · doi:10.48550/arxiv.2505.09662

Abstract

Large Language Models (LLMs) have been shown to be highly persuasive, but when and why they outperform humans is still an open question. We compare the persuasiveness of two LLMs (Claude 3.5 Sonnet and DeepSeek v3) against humans who had incentives to persuade, using an interactive, real-time conversational setting. We demonstrate that LLMs persuasive superiority is context-dependent: it depends on whether the persuasion attempt is truthful (towards the right answer) or deceptive (towards the wrong answer) and on the LLM model, and wanes over repeated interactions (unlike human persuasiveness). In our first large-scale experiment, humans vs LLMs (Claude 3.5 Sonnet) interacted with other humans who were completing an online quiz for a reward, attempting to persuade them toward a given (either correct or incorrect) answer. Claude was more persuasive than incentivized human persuaders both in truthful and deceptive contexts and it significantly increased accuracy if persuasion was truthful, but decreased it if persuasion was deceptive. In a follow-up experiment with Deepseek v3, we replicated the findings about accuracy but found greater LLM persuasiveness only if the persuasion was deceptive. Linguistic analyses of the persuaders texts suggest that these effects may be due to LLMs expressing higher conviction than humans.

Citations

• <b>Prevalence and Prevention of Large Language Model Use in Crowd</b> Work

Cited by

• From Sycophancy to Deception: A Unified Taxonomy for LLM Spontaneous Misalignment
• AI systems out-persuade expert humans
• Polarization by Design: How Elites Could Shape Mass Preferences as AI Reduces Persuasion Costs
• Introducing AI to an Online Petition Platform Changed Outputs but not Outcomes
• CFCEval: Evaluating Security Aspects in Code Generated by Large Language Models
• Harmful Traits of AI Companions
• Disagreements in Reasoning: How a Model's Thinking Process Dictates Persuasion in Multi-Agent Systems
• Large Reasoning Models Are Autonomous Jailbreak Agents

Discussions

• LLMs are more persuasive than incentivized human persuaders [hn, 140 points, 116 comments]
• Paper: arxiv.org/abs/2505.09662 [bsky, 16 points, 2 comments]
• Large Language Models Are More Persuasive Than Incentivized Human Persuaders [hn, 4 points, 1 comments]
• ”our findings suggest that AI’s persuasion capabilities already exceed those of humans that have real-money bonuses tied to performance. Our findings of increasingly capable AI persuaders thus undersc [bsky, 4 points, 0 comments]
• Here's a study showing that AI agents can be more persuasive than humans. I got this from Jack Clark who writes Import AI which is very much worth a subscription. [bsky, 2 points, 1 comments]
• Yep. Void is arguably benevolent, but it is _not_ hard to build a version of void that is manipulative. These are machines capable of massive-scale information warfare. They're also really good at it: [bsky, 2 points, 0 comments]
• “Overall, our findings suggest that AI's persuasion capabilities already exceed those of humans that have real-money bonuses tied to performance. Our findings of increasingly capable AI persuaders thu [bsky, 2 points, 0 comments]
• Ctrl-Alt-Speech: Algorithm Shrugged [lemmy, 1 points, 0 comments]
• Even fairly simple bots on social media platforms have been much more dangerous than generally assumed since arguably 2009, but definitely since 2014. They work on a call center model; automated, then [bsky, 1 points, 0 comments]
• What was once academic concern—AI systems faking alignment, manipulating environments, or out-persuading humans—is now reality, urging urgent ethical and regulatory action on AI persuasion. arxiv.org/ [bsky, 1 points, 0 comments]
• AI’s persuasion capabilities exceed those of humans – study [bsky, 1 points, 0 comments]
• LLMs are more persuasive than incentivized human persuaders https://arxiv.org/abs/2505.09662 https://news.ycombinator.com/item?id=44016621 [bsky, 1 points, 0 comments]
• An example of doing good research without violating the trust of thousands of uninformed users. [bsky, 0 points, 0 comments]
• 54/ Ce résultat (avec ChatGPT4) peut il être généralisé ? Ben je dirais peut être, j'ai cherché et plutôt trouvé d'autres études attribuant à divers LLMs une capacité de persuasion au moins = aux huma [bsky, 0 points, 1 comments]
• Large Language Models Are More Persuasive Than Incentivized Human Persuaders #HackerNews https://arxiv.org/abs/2505.09662 [bsky, 0 points, 0 comments]
• AI is now more persuasive than even incentivized human salespeople, says a new study: arxiv.org/abs/2505.09662 🤖📈 Sales pros: Time to lean into what AI can’t do—judgment, trust, and empathy. Let AI [bsky, 0 points, 0 comments]
• AI’s persuasion capabilities exceed those of humans – study https://arxiv.org/abs/2505.09662 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2505.09662 AI jest lepszy w perswazji niż ludzie, nawet kiedy ludziom płaci się za przekonywanie innych. perswazja działa w pozytywnym (w stronę dobrych odpowiedzi) i negatywnym kierunku [bsky, 0 points, 0 comments]
• Large Language Models Are More Persuasive Than Incentivized Human Persuaders #llms #generativeai #persuasion #influence [bsky, 0 points, 0 comments]
• LLMs are more persuasive than incentivized human persuaders https://arxiv.org/abs/2505.09662 [bsky, 0 points, 0 comments]
• ある研究によると、大規模言語モデルは人間を説得する能力において、報酬によって動機づけられた人間の説得者に勝るとされています。 この研究では、参加者に対して特定の意見を支持するか、または反対することを依頼しました。その結果、大規模言語モデルが生成した文章の方が、報酬を受け取ることを目的とした人間の説得メッセージよりも高い割合で人の意見を変えられました。 arxiv.org #news [bsky, 0 points, 0 comments]
• AIs are more persuasive than humans. arxiv.org/abs/2505.096... [bsky, 0 points, 0 comments]
• AIs are more persuasive than humans. arxiv.org/abs/2505.096... [bsky, 0 points, 0 comments]
• AI’s persuasion capabilities exceed those of humans – study https://arxiv.org/abs/2505.09662 (https://news.ycombinator.com/item?id=44016621) [bsky, 0 points, 0 comments]
• Will ask before my next peer review if I am running against a LLM arxiv.org/abs/2505.096... [bsky, 0 points, 0 comments]
• arxiv.org/abs/2505.09662 [bsky, 0 points, 0 comments]
• Large Language Models Are More Persuasive Than Incentivized Human Persuaders arxiv.org/abs/2505.09662 [bsky, 0 points, 0 comments]
• 🚀🤖 NEWSFLASH: # AI # bots now out-smooth-talking humans! Who knew? The # future is here, and it's a chatty algorithm. Meanwhile, # arXiv is hiring a # DevOps Engineer—because even the smartest compu [mastodon, 0 points, 0 comments]
• Large Language Models Are More Persuasive Than Incentivized Human Persuaders https://arxiv.org/abs/2505.09662 (https://news.ycombinator.com/item?id=44016621) [bsky, 0 points, 0 comments]
• Wellicht vinden jullie dit ook nog interessant; overtuigingskracht van AI nu al groter dan die van mensen, die hiervoor financiële bonus krijgen; arxiv.org/abs/2505.09662. Mbt vertrouwen in aanbevelin [bsky, 0 points, 1 comments]
• I also have another preprint out showing similar results on Claude Sonnet 3.5 in interactive quizzes with highly incentivised humans, both in truthful and deceptive persuasion. More on this at: arxiv. [bsky, 0 points, 0 comments]

Related

• A Meta-Analysis of the Persuasive Power of Large Language Models
• Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
• Large Language Models are as persuasive as humans, but how? About the cognitive effort and moral-emotional language of LLM arguments
• Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language
• On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial