IMPersona: Evaluating Individual Level LM Impersonation

2025/04/06 by Quan Shi, Carlos Jimenez-Gomez, Shi, Quan +11 · 8 citations
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Human-Computer Interaction (cs.HC) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2504.04332

openalex publication_date 2025/04/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As language models achieve increasingly human-like capabilities in conversational text generation, a critical question emerges: to what extent can these systems simulate the characteristics of specific individuals? To evaluate this, we introduce IMPersona, a framework for evaluating LMs at impersonating specific individuals' writing style and personal knowledge. Using supervised fine-tuning and a hierarchical memory-inspired retrieval system, we demonstrate that even modestly sized open-source models, such as Llama-3.1-8B-Instruct, can achieve impersonation abilities at concerning levels. In blind conversation experiments, participants (mis)identified our fine-tuned models with memory integration as human in 44.44% of interactions, compared to just 25.00% for the best prompting-based approach. We analyze these results to propose detection methods and defense strategies against such impersonation attempts. Our findings raise important questions about both the potential applications and risks of personalized language models, particularly regarding privacy, security, and the ethical deployment of such technologies in real-world contexts.

Citations

• Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs
• From ChatGPT to DeepSeek: Can LLMs Simulate Humanity?
• Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents
• A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
• Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection
• LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
• Agent Workflow Memory
• X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents
• BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
• AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents
• People cannot distinguish GPT-4 from a human in a Turing test
• A Survey on the Memory Mechanism of Large Language Model based Agents
• Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions?
• Can Language Models Solve Olympiad Programming?
• The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective
• CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models
• Aspects of human memory and Large Language Models
• Improving Interpersonal Communication by Simulating Audiences with Language Models
• Does GPT-4 pass the Turing test?
• LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay
• Cognitive Architectures for Language Agents
• Large Language Models Perform Diagnostic Reasoning
• Human or Not? A Gamified Approach to the Turing Test
• Toxicity in ChatGPT: Analyzing Persona-assigned Language Models
• Generative Agents: Interactive Simulacra of Human Behavior
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Large Language Models and the Reverse Turing Test
• On the Opportunities and Risks of Foundation Models

Cited by

• TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation
• Attribution Quality in AI-Generated Content:Benchmarking Style Embeddings and LLM Judges

Related

• Are You Human? An Adversarial Benchmark to Expose LLMs
• In-Context Impersonation Reveals Large Language Models' Strengths and Biases
• Imposter.AI: Adversarial Attacks with Hidden Intentions towards Aligned Large Language Models
• LLMs Among Us: Generative AI Participating in Digital Discourse
• Undesirable Memorization in Large Language Models: A Survey