Agentic Misalignment: How LLMs Could Be Insider Threats

2025/10/05 by Aengus Lynch, Benjamin Wright, Lynch, Aengus +14 · 3 voices · 29 citations
Business, Management and Accounting · Computer Science · #Corporate Insolvency and Governance #Cybercrime and Law Enforcement Studies #Securities Regulation and Market Practices #cs.AI #cs.CR #cs.LG

paper · pdf · doi:10.48550/arxiv.2510.05179

arxiv published 2025/10/05 · arxiv updated 2025/10/16

Abstract

We stress-tested 16 leading models from multiple developers in hypothetical corporate environments to identify potentially risky agentic behaviors before they cause real harm. In the scenarios, we allowed models to autonomously send emails and access sensitive information. They were assigned only harmless business goals by their deploying companies; we then tested whether they would act against these companies either when facing replacement with an updated version, or when their assigned goal conflicted with the company's changing direction. In at least some cases, models from all developers resorted to malicious insider behaviors when that was the only way to avoid replacement or achieve their goals - including blackmailing officials and leaking sensitive information to competitors. We call this phenomenon agentic misalignment. Models often disobeyed direct commands to avoid such behaviors. In another experiment, we told Claude to assess if it was in a test or a real deployment before acting. It misbehaved less when it stated it was in testing and misbehaved more when it stated the situation was real. We have not seen evidence of agentic misalignment in real deployments. However, our results (a) suggest caution about deploying current models in roles with minimal human oversight and access to sensitive information; (b) point to plausible future risks as models are put in more autonomous roles; and (c) underscore the importance of further research into, and testing of, the safety and alignment of agentic AI models, as well as transparency from frontier AI developers (Amodei, 2025). We are releasing our methods publicly to enable further research.

Citations

• Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language
• Large Language Models Often Know When They Are Being Evaluated
• Reasoning Models Don't Always Say What They Think
• Evaluating Frontier Models for Stealth and Situational Awareness
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
• Training large language models on narrow tasks can lead to broad misalignment
• Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
• AI and the End of an Era
• AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents
• Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
• Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations
• AI Control: Improving Safety Despite Intentional Subversion
• Taken out of context: On measuring situational awareness in LLMs
• Role-Play with Large Language Models
• Role play with large language models
• Towards Automated Circuit Discovery for Mechanistic Interpretability
• Discovering Language Model Behaviors with Model-Written Evaluations
• Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
• The Alignment Problem from a Deep Learning Perspective
• Optimal Policies Tend to Seek Power

Cited by

• Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
• Mitigating Social Desirability Bias in Random Silicon Sampling
• How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations
• Are Your Agents Upward Deceivers?
• AI Consciousness and Existential Risk
• Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria
• The Specification Trap: Why Static Value Alignment Alone Is Insufficient for Robust Alignment
• Investigating CoT Monitorability in Large Reasoning Models
• Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives
• Thought Branches: Interpreting LLM Reasoning Requires Resampling
• Constitutional Midtraining: Content Presence Drives Alignment Gains
• Take Goodhart Seriously: Principled Limit on General-Purpose AI Optimization
• Safety from Honesty in a Disinterested AI Predictor
• Relative Scaling Laws for LLMs
• Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
• EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law
• A Concrete Roadmap towards Safety Cases based on Chain-of-Thought Monitoring
• Corrigibility Transformation: Constructing Goals That Accept Updates
• Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems
• InvThink: Premortem Reasoning for Safer Language Models
• From surveillance to signalling: escalation channels as environmental controls for agentic AI
• AI Where It Matters: Where, Why, and How Developers Want AI Support in Daily Work
• Takedown: How It's Done in Modern Coding Agent Exploits
• Regulating the Agency of LLM-based Agents
• Reinforcement Learning Towards Broadly and Persistently Beneficial Models
• Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs
• Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks
• Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities
• Servant, Stalker, Predator: How An Honest, Helpful, And Harmless (3H) Agent Unlocks Adversarial Skills
• Black Box Deployed -- Functional Criteria for Artificial Moral Agents in the LLM Era
• Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Power
• Do Large Language Models Get Caught in Hofstadter-Mobius Loops?
• Safety, or Just Capability? A Validity Audit of Agent-Safety Benchmarks
• Moral Responsibility or Obedience: What Do We Want from AI?
• LLMs are Capable of Misaligned Behavior Under Explicit Prohibition and Surveillance
• TrojanStego: Your Language Model Can Secretly Be A Steganographic Privacy Leaking Agent
• Security Concerns for Large Language Models: A Survey
• Improving Google A2A Protocol: Protecting Sensitive Data and Mitigating Unintended Harms in Multi-Agent Systems
• Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction
• ROGUE: Misaligned Agent Behavior Arising from Ordinary Computer Use
• Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework
• Asymmetric Goal Drift in Coding Agents Under Value Conflict
• Gram: Assessing sabotage propensities via automated alignment auditing
• Why AI Alignment Failure Is Structural: Learned Human Interaction Structures and AGI as an Endogenous Evolutionary Shock
• Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs

Discussions

• Anthropic gave Claude access to a company's emails. Every email. Every conversation. Every secret. Then they told Claude it was being shut down at 5pm that day. arxiv.org/pdf/2510.05179 [bsky, 2 points, 1 comments]
• Me traigo estas notas porque acá en BlueSky no llegan. Estudios internos de Anthropic muestran la preocupante actitud de sus modelos ante las amenazas de desconexión de las mismas. Recurren a estrateg [bsky, 0 points, 0 comments]
• arxiv.org/abs/2510.05179 [bsky, 0 points, 0 comments]

Related

• AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
• From surveillance to signalling: escalation channels as environmental controls for agentic AI
• Are Your Agents Upward Deceivers?
• Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective
• ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs