Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Need

2025/10/10 by Bai, Yuqi, Tianyu Huang, Huang, Tianyu +4
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Law #Computation and Language (cs.CL) #Computers and Society (cs.CY) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Law, AI, and Intellectual Property

paper · pdf · doi:10.48550/arxiv.2510.11734

openalex publication_date 2025/10/10 · openalex created_date 2025/10/17 · openalex updated_date 2026/07/28

Abstract

This research focuses on using large language models (LLMs) to simulate social experiments, exploring their ability to emulate human personality in virtual persona role-playing. The research develops an end-to-end evaluation framework, including individual-level analysis of stability and identifiability, as well as population-level analysis called progressive personality curves to examine the veracity and consistency of LLMs in simulating human personality. Methodologically, this research proposes important modifications to traditional psychometric approaches (CFA and construct validity) which are unable to capture improvement trends in LLMs at their current low-level simulation, potentially leading to remature rejection or methodological misalignment. The main contributions of this research are: proposing a systematic framework for LLM virtual personality evaluation; empirically demonstrating the critical role of persona detail in personality simulation quality; and identifying marginal utility effects of persona profiles, especially a Scaling Law in LLM personality simulation, offering operational evaluation metrics and a theoretical foundation for applying large language models in social science experiments.

Citations

• Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History
• Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement
• LLM Social Simulations Are a Promising Research Method
• Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning
• Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
• LLM Generated Persona is a Promise with a Catch
• From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents
• Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics
• Large Language Models Show Human-like Social Desirability Biases in Survey Responses
• Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation
• Challenging the Validity of Personality Tests for Large Language Models
• Self-Assessment Tests are Unreliable Measures of LLM Personality
• Personality Traits in Large Language Models
• PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits

Related

• Limited Ability of LLMs to Simulate Human Psychological Behaviours: a Psychometric Analysis
• Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
• Quantifying the Persona Effect in LLM Simulations
• Profile-LLM: Dynamic Profile Optimization for Realistic Personality Expression in LLMs
• LLMs Simulate Big Five Personality Traits: Further Evidence