LLM Generated Persona is a Promise with a Catch

2025/03/18 by Ang Li, H.F. Chen, Li, Ang +5 · 42 citations
Computer Science · Engineering · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #Digital Economy and Work Transformation #FOS: Computer and information sciences #Persona Design and Applications #Social and Information Networks (cs.SI) #Transportation and Mobility Innovations

paper · pdf · doi:10.48550/arxiv.2503.16527

openalex publication_date 2025/03/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The use of large language models (LLMs) to simulate human behavior has gained significant attention, particularly through personas that approximate individual characteristics. Persona-based simulations hold promise for transforming disciplines that rely on population-level feedback, including social science, economic analysis, marketing research, and business operations. Traditional methods to collect realistic persona data face significant challenges. They are prohibitively expensive and logistically challenging due to privacy constraints, and often fail to capture multi-dimensional attributes, particularly subjective qualities. Consequently, synthetic persona generation with LLMs offers a scalable, cost-effective alternative. However, current approaches rely on ad hoc and heuristic generation techniques that do not guarantee methodological rigor or simulation precision, resulting in systematic biases in downstream tasks. Through extensive large-scale experiments including presidential election forecasts and general opinion surveys of the U.S. population, we reveal that these biases can lead to significant deviations from real-world outcomes. Our findings underscore the need to develop a rigorous science of persona generation and outline the methodological innovations, organizational and institutional support, and empirical foundations required to enhance the reliability and scalability of LLM-driven persona simulations. To support further research and development in this area, we have open-sourced approximately one million generated personas, available for public access and analysis at https://huggingface.co/datasets/Tianyi-Lab/Personas.

Cited by

• LLM Personas as a Substitute for Field Experiments in Method Benchmarking
• Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novels
• Do Persona-Infused LLMs Affect Performance in a Strategic Reasoning Game?
• CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models
• Understanding Down Syndrome Stereotypes in LLM-Based Personas
• Misalignment of LLM-Generated Personas with Human Perceptions in Low-Resource Settings
• Benchmarking In-context Experiential Learning Through Repeated Product Recommendations
• Efficient Inference Using Large Language Models with Limited Human Data: Fine-Tuning then Rectification
• Predicting Social Media User Actions: A Hybrid Approach for Common and Rare Behavior Prediction on Bluesky
• DeepPersona: A Generative Engine for Scaling Deep Synthetic Personas
• Do Androids Dream of Unseen Puppeteers? Probing for a Conspiracy Mindset in Large Language Models
• Graph-Based Alternatives to LLMs for Human Simulation
• Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
• Simulating and Experimenting with Social Media Mobilization Using LLM Agents
• Agentic Economic Modeling
• When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses
• Argument Collapse: LLMs Flatten Long-Form Public Debate
• Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts
• Race and Gender in LLM-Generated Personas: A Large-Scale Audit of 41 Occupations
• Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification
• Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Need
• The Prompt Engineering Report Distilled: Quick Start Guide for Life Sciences
• When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
• Leveraging LLMs for Persona-Based Visualization of Election Data
• Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation
• SYNTHIA: Synthetic Yet Naturally Tailored Human-Inspired PersonAs
• LLM-Based Community Surveys for Operational Decision Making in Interconnected Utility Infrastructures
• Temperature and Persona Shape LLM Agent Consensus With Minimal Accuracy Gains in Qualitative Coding
• Auto-Formulating Dynamic Programming Problems with Large Language Models
• Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity
• LLM-Based Social Simulations Require a Boundary
• Leveraging Interview-Informed LLMs to Model Survey Responses: Comparative Insights from AI-Generated and Human Data
• Aligning LLM with human travel choices: a persona-based embedding learning approach
• Twin-2K-500: A dataset for building digital twins of over 2,000 people based on their answers to over 500 questions
• Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions
• Images Amplify Misinformation Sharing in Vision-Language Models
• Social Theory Should Be a Structural Prior for Agentic AI: A Formal Framework for Multi-Agent Social Systems
• Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Prompts
• LLM-Evaluation Tropes: Perspectives on the Validity of LLM-Evaluations
• Emulate or Estimate? The Divergent Strengths of Base and Post-Trained Language Models for Opinion Simulation
• Studying People to Study AI: Expert Perspectives on the Epistemic Fit and Barriers of Human Research in AI Safety & Ethics
• How Is Generative AI Used for Persona Development?: A Systematic Review of 52 Research Articles

Related

• Population-Aligned Persona Generation for LLM-based Social Simulation
• German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies
• Quantifying the Persona Effect in LLM Simulations
• Evaluating the Effectiveness of Persona Simulation in Opinion Prediction with GPT-4.1
• Evaluating the Effectiveness of Persona Simulation in Opinion Prediction with GPT-4.1