PRISON: Unmasking the Criminal Potential of Large Language Models
2025/06/19 by Xinyi Wu, Geng Hong, Wu, Xinyi +9
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Cryptography and Security (cs.CR) #Deception detection and forensic psychology #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Mental Health via Writing
paper · pdf · doi:10.48550/arxiv.2506.16150
openalex publication_date 2025/06/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Abstract
As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessment of their criminal capability in realistic interactions. We propose a unified framework PRISON, to quantify LLMs' criminal potential across five traits: False Statements, Frame-Up, Psychological Manipulation, Emotional Disguise, and Moral Disengagement. Using structured crime scenarios adapted from classic films grounded in reality, we evaluate both criminal potential and anti-crime ability of LLMs. Results show that state-of-the-art LLMs frequently exhibit emergent criminal tendencies, such as proposing misleading statements or evasion tactics, even without explicit instructions. Moreover, when placed in a detective role, models recognize deceptive behavior with only 44% accuracy on average, revealing a striking mismatch between conducting and detecting criminal behavior. These findings underscore the urgent need for adversarial robustness, behavioral alignment, and safety mechanisms before broader LLM deployment.
Citations
- The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
- Mitigating Deceptive Alignment via Self-Monitoring
- The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
- Towards LLMs Robustness to Changes in Prompt Format Styles
- VEGAS: Towards Visually Explainable and Grounded Artificial Social Intelligence
- Exploring Large Language Models for Word Games:Who is the Spy?
- CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulation
- Human Decision-making is Susceptible to AI-driven Manipulation
- International AI Safety Report
- MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environments
- Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models
- A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios
- Persuasion with Large Language Models: A Survey of Empirical Evidence, Study Methodologies, and Ethical Implications
- On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback
- EgoSocialArena: Benchmarking the Social Intelligence of Large Language Models from a First-person Perspective
- AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents
- DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
- AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
- Truth is Universal: Robust Detection of Lies in LLMs
- BeHonest: Benchmarking Honesty in Large Language Models
- InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
- Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant
- Resistance Against Manipulative AI: key factors and possible actions
- Monotonic Paraphrasing Improves Generalization of Language Model Prompting
- Evaluating Frontier Models for Dangerous Capabilities
- Embodied LLM Agents Learn to Cooperate in Organized Teams
- Can LLM-Augmented autonomous agents cooperate?, An evaluation of their cooperative capabilities through Melting Pot
- Academically intelligent LLMs are not necessarily socially intelligent
- Enhance Reasoning for Large Language Models in the Game Werewolf
- CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents
- Honesty Is the Best Policy: Defining and Mitigating AI Deception
- Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games
- Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities
- Large Language Models can Strategically Deceive their Users when Put Under Pressure
- Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
- AI Deception: A Survey of Examples, Risks, and Potential Solutions
- ProAgent: Building Proactive Cooperative Agents with Large Language Models
- Better Zero-Shot Reasoning with Role-Play Prompting
- Evaluating the Moral Beliefs Encoded in LLMs
- Explore, Establish, Exploit: Red Teaming Language Models from Scratch
- Benchmarking Foundation Models with Language-Model-as-an-Examiner
- Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science
- Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
- Evaluating large language models in theory of mind tasks
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- Evaluating and Inducing Personality in Pre-trained Language Models
- Training language models to follow instructions with human feedback
- Language Models are Few-Shot Learners
- Psychologically based Virtual-Suspect for Interrogative Interview\n Training
- Reliability, Validity, and Predictive Utility of the 25-item Criminogenic Cognitions Scale (CCS)
- An Assessment for Criminal Thinking
- A Coefficient of Agreement for Nominal Scales
Related