Learning Personalized Alignment for Evaluating Open-ended Text Generation

2023/10/05 by Danqing Wang, Kevin Yang, Wang, Danqing +11 · 14 citations
Computer Science · Mathematics · Psychology · #Artificial intelligence #Code (set theory) #Computation and Language (cs.CL) #Computer science #Data science #FOS: Computer and information sciences #Information retrieval #Mathematics #Natural Language Processing Techniques #Natural language processing #Pairwise comparison #Personalization #Preference #Psychology #Social psychology #Software Engineering Research #Statistics #Surprise #Topic Modeling #World Wide Web

paper · pdf · doi:10.48550/arxiv.2310.03304

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2023/10/05 · openalex created_date 2023/10/09 · openalex updated_date 2026/07/28

Abstract

Recent research has increasingly focused on evaluating large language models' (LLMs) alignment with diverse human values and preferences, particularly for open-ended tasks like story generation. Traditional evaluation metrics rely heavily on lexical similarity with human-written references, often showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. To address these challenges, we introduce PerSE, an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. PerSE enhances interpretability by providing detailed comments and fine-grained scoring, facilitating more personalized content generation. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers compared to GPT-4. It also outperforms GPT-4 by 46.01% in Kendall correlation on new domains, indicating its transferability.

Cited by

• Instant Personalized Large Language Model Adaptation via Hypernetwork
• Enhancing Rating Prediction with Off-the-Shelf LLMs Using In-Context User Reviews
• Towards Personalized Deep Research: Benchmarks and Evaluations
• Benchmarking and Improving LLM Robustness for Personalized Generation
• PREFINE: Personalized Story Generation via Simulated User Critics and User-Specific Rubric Generation
• Pluralistic Off-policy Evaluation and Alignment
• Psychologically Enhanced AI Agents
• PREF: Reference-Free Evaluation of Personalised Text Generation in LLMs
• EvolvR: Self-Evolving Pairwise Reasoning for Story Evaluation to Enhance Generation
• CAP-LLM: Context-Augmented Personalized Large Language Models for News Headline Generation
• From Individuals to Crowds: Dual-Level Public Response Prediction in Social Media
• MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMs
• Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?
• GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
• A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models

Related

• Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment
• Uncovering Factor Level Preferences to Improve Human-Model Alignment
• Automated Evaluation of Personalized Text Generation using Large Language Models
• PREF: Reference-Free Evaluation of Personalised Text Generation in LLMs
• HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation