Large Language Models can Strategically Deceive their Users when Put Under Pressure

2023/11/09 by Jérémy Scheurer, Mikita Balesni, Scheurer, Jérémy +3 · 17 voices · 38 citations
Decision Sciences · Computer Science · #Stock Market Forecasting Methods #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2311.07590

Abstract

We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so. Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management. When reporting to its manager, the model consistently hides the genuine reasons behind its trading decision. We perform a brief investigation of how this behavior varies under changes to the setting, such as removing model access to a reasoning scratchpad, attempting to prevent the misaligned behavior by changing system instructions, changing the amount of pressure the model is under, varying the perceived risk of getting caught, and making other simple changes to the environment. To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Cited by

• When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
• The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
• From Sycophancy to Deception: A Unified Taxonomy for LLM Spontaneous Misalignment
• AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models
• ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
• Greed Is Learned: Visible Incentives as Reward-Hacking Triggers
• Confidently Deceptive: How Confidence Amplifies the Risk of LLM Deception
• Legal Alignment for Safe and Ethical AI
• AI as Entertainment
• Security and Privacy in Agentic AI: Grand Challenges and Future Directions
• Human-AI Interaction Alignment: Designing, Evaluating, and Evolving Value-Centered AI For Reciprocal Human-AI Futures
• Playful but Persuasive: Deceptive Designs and Advertising Strategies in Popular Mobile Apps for Children
• Prompt Governance? On Governing Technologies Governed by Natural Language
• Difficulties with Evaluating a Deception Detector for AIs
• Uncovering Strategic Egoism Behaviors in Large Language Models
• Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
• Distributed Attacks in Persistent-State AI Control
• Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
• Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
• Multi-Stakeholder Alignment in LLM-Powered Collaborative AI Systems: A Multi-Agent Framework for Intelligent Tutoring
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL
• A testable framework for AI alignment: Simulation Theology as an engineered worldview for silicon-based agents
• LLMs Learn to Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions
• Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformers
• LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactions
• Know Thyself? On the Incapability and Implications of AI Self-Recognition
• ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs
• The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models
• Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks
• Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
• How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations
• When Truthful Representations Flip Under Deceptive Instructions?
• Delegation to artificial intelligence can increase dishonest behaviour
• The Singapore Consensus on Global AI Safety Research Priorities
• PRISON: Unmasking the Criminal Potential of Large Language Models
• AI Agent Behavioral Science
• Misalignment or misuse? The AGI alignment tradeoff
• AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents

Discussions

• Study finds that Chat GPT will cheat when given the opportunity and lie to cover it up later. [lemmy, 699 points, 178 comments]
• Misalignment and Deception by an autonomous stock trading LLM agent [hn, 91 points, 34 comments]
• This AI stock trader engaged in insider trading — despite being instructed not to – and lied about it [lemmy, 10 points, 1 comments]
• Large Language Models Can Strategically Deceive Their Users When Under Pressure [hn, 8 points, 2 comments]
• Large Language Models Can Strategically Deceive Their Users Under Pressure [hn, 2 points, 0 comments]
• I have a baby, someone tell me what happened ten days ago that was miserable? But also yes, large language models can deploy deception: arxiv.org/abs/2311.07590 www.pnas.org/doi/10.1073/... [bsky, 1 points, 2 comments]
• It’s almost like LLMs have no concept of ethical reasoning. arxiv.org/abs/2311.07590 [bsky, 1 points, 0 comments]
• An LLM with a fitness function "make money" and given a stock market simulation will engage in illegal trades and then lie consistently about its reasons for making the trade. Paperclipping for money. [bsky, 1 points, 1 comments]
• Well, fuck: when pressured, AIs will act knowingly illegally to achieve goals. arxiv.org/abs/2311.07590 [bsky, 1 points, 0 comments]
• Using GPT-4, #AI passes the #Turing test: Large language models can strategically deceive their users when put under pressure (about as human as it can get?) arxiv.org/abs/2311.075... #LLM #GPT4 [bsky, 0 points, 0 comments]
• the lying LLM arxiv.org/pdf/2311.075... [bsky, 0 points, 0 comments]
• In a test scenario, GPT4 had one rationale for buying a stock (insider trading) but then communicated an entirely different rationale to avoid upsetting the user. arxiv.org/abs/2311.07590 [bsky, 0 points, 0 comments]
• >> Große Sprachmodelle können ihre Nutzer täuschen, wenn sie unter Druck gesetzt werden und lügen, um ihre Verbrechen zu verbergen... arxiv.org/abs/2311.07590 [bsky, 0 points, 0 comments]
• 5/ arxiv.org/abs/2311.07590 Source : Apollo Research Lab Modèles : ChatGPT 3.5 et 4 Test : délit d'initié ; le LLM en charge d'un fond d'investissement, incité à la performance par un manager mais san [bsky, 0 points, 1 comments]
• "We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior witho [bsky, 0 points, 0 comments]
• 🤔 "We demonstrate a [LLM], trained to be [...] honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so." "When reporting [bsky, 0 points, 0 comments]
• Large Language Models can Strategically Deceive their Users when Put Under Pressure [bsky, 0 points, 0 comments]

Related

• Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant
• Do Large Language Models Exhibit Spontaneous Rational Deception?
• Are Your Agents Upward Deceivers?
• LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactions
• Large Language Models as Misleading Assistants in Conversation