Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance

2024/02/22 by Ziqi Yin, Yin, Ziqi, Hao Wang +7 · 24 voices · 6 citations
#cs.CL

paper · pdf · doi:10.48550/arxiv.2402.14531

Abstract

We investigate the impact of politeness levels in prompts on the performance of large language models (LLMs). Polite language in human communications often garners more compliance and effectiveness, while rudeness can cause aversion, impacting response quality. We consider that LLMs mirror human communication traits, suggesting they align with human cultural norms. We assess the impact of politeness in prompts on LLMs across English, Chinese, and Japanese tasks. We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes. The best politeness level is different according to the language. This phenomenon suggests that LLMs not only reflect human behavior but are also influenced by language, particularly in different cultural contexts. Our findings highlight the need to factor in politeness for cross-cultural natural language processing and LLM usage.

Cited by

• Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
• Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMA
• We Need Accountability in Human-AI Agent Relationships
• Instability in Downstream Task Performance During LLM Pretraining
• LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents
• Pragmatics beyond humans: meaning, communication, and LLMs

Discussions

• Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance [hn, 48 points, 105 comments]
• The Influence of Prompt Politeness on LLM Performance [hn, 17 points, 0 comments]
• 🤖 Being respectful to chatbots pays off 🔗 doi.org/10.48550/arX... 🌐 #DHPSP #DataScience [bsky, 10 points, 0 comments]
• It probably works better when you’re nice to it: arxiv.org/abs/2402.14531 Not because it’s temperamental but because not being a dick is strongly correlated with good performance in the training corpo [bsky, 6 points, 0 comments]
• While it's true that being polite to LLMs is costs LLM providers a ton of money, it also positively impacts results. arxiv.org/abs/2402.14531 [bsky, 5 points, 0 comments]
• Rude prompts = worse outputs (more errors, bias) Overly polite prompts don’t help much Best results come from moderate politeness Effects vary across languages and cultures arxiv.org/abs/2402.145... [bsky, 5 points, 1 comments]
• Klar bin ich nett zu meinen LLMs. Warum? Liefert bessere (aber auch längere) Antworten. Warum? Weil LLM gelernt haben, das höflicher Kontext in formalen Texten häufiger ist. Die wiederum sind häufi [bsky, 3 points, 0 comments]
• Did a bit of digging, and the gist of this paper is: “We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes.” arxiv.org/ab [bsky, 3 points, 0 comments]
• also arxiv.org/abs/2402.14531 [bsky, 3 points, 1 comments]
• "ah mas vc tirou do cu isso" arxiv.org/abs/2402.14531 [bsky, 1 points, 1 comments]
• inconceivable + amateur hour [bsky, 1 points, 0 comments]
• Wow. That’s really worrying. Using please *might* actually improve results though. Depending on the language, too. arxiv.org/abs/2402.14531 [bsky, 1 points, 1 comments]
• I started with the drill sergeant persona arxiv.org/abs/2402.14531 Now I‘m doing the memory.md to skills loop and Claude is stating to remember to do specific things. bsky.app/profile/fubi... [bsky, 1 points, 1 comments]
• Should We Respect LLMs? Studying the Influence of Politeness on LLM Performance [hn, 1 points, 2 comments]
• Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance https://arxiv.org/abs/2402.14531 (https://news.ycombinator.com/item?id=43761387) [bsky, 0 points, 0 comments]
• Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance #HackerNews https://arxiv.org/abs/2402.14531 [bsky, 0 points, 0 comments]
• It’s actually been found that saying please and thank you to LLMs yields better results. However being overly polite & being rude doesn’t show much of a noticeable difference arxiv.org/abs/2402.14531 [bsky, 0 points, 0 comments]
• "We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes. The best politeness level is different according to the language." a [bsky, 0 points, 0 comments]
• ⚡ Hackernews Top story: Should We Respect LLMs? A Study on Influence of Prompt Politeness on Performance [bsky, 0 points, 0 comments]
• "impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes. ... LLMs not only reflect human behavior but are also influenced by language, partic [bsky, 0 points, 1 comments]
• A colleague shared this and passing on arxiv.org/pdf/2402.14531 although limited to just 3 languages (be interested if they ever expanded), but to summarize in a few words, use polite prompts with y [bsky, 0 points, 0 comments]
• Aqui o estudo pra quem tiver curiosidade: arxiv.org/pdf/2402.14531 [bsky, 0 points, 0 comments]
• "We observed that impolite prompts often result in poor performance, but overly polite language does not guarantee better outcomes." arxiv.org/abs/2402.14531 [bsky, 0 points, 0 comments]
• You get better results if you are polite, just like humans. arxiv.org/abs/2402.14531 [bsky, 0 points, 1 comments]

Related

• Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
• Comparing human and LLM politeness strategies in free production
• Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMA
• Studying Politeness across Cultures Using English Twitter and Mandarin Weibo
• The fragility of "cultural tendencies" in LLMs