One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity

2024/11/07 by Sonia K. Murthy, Tomer Ullman, Murthy, Sonia K. +3 · 3 voices · 28 citations
Computer Science · #Biology #Computer science #Diversity (politics) #Fish <Actinopterygii> #Fishery #Natural Language Processing Techniques #Sociology #Topic Modeling

paper · pdf · doi:10.18653/v1/2025.naacl-long.561

openalex publication_date 2025/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Sonia Krishna Murthy, Tomer Ullman, Jennifer Hu. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Cited by

• PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents
• When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play
• A Diagnostic Framework for AI Agent Behavior
• More Is Not More: What Matters for Diversity in LLM Opinions?
• Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
• Epistemic Diversity and Knowledge Collapse in Large Language Models
• Cognitive models can reveal interpretable value trade-offs in language models
• Base Models Beat Aligned Models at Randomness and Creativity
• LLM Social Simulations Are a Promising Research Method
• What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
• Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies
• Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
• This human study did not involve human subjects: Validating LLM simulations as behavioral evidence
• Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety
• Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization
• Post-training Large Language Models for Diverse High-Quality Responses
• Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI
• SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents
• Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles
• IDRBench: Understanding the Capability of Large Language Models on Interdisciplinary Research
• LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
• From Prompts to Constructs: A Dual-Validity Framework for LLM Research in Psychology
• Diversity-Aware Policy Optimization for Large Language Model Reasoning
• Dynamic Risk Assessments for Offensive Cybersecurity Agents
• What LLMs Think When You Don't Tell Them What to Think About?
• The Geometry of Self-Verification in a Task-Specific Reasoning Model
• Signatures of human-like processing in Transformer forward passes
• Evaluating the Diversity and Quality of LLM Generated Content

Discussions

• (1/9) Excited to share my recent work on "Alignment reduces LM's conceptual diversity" with @tomerullman.bsky.social and @jennhu.bsky.social, to appear at #NAACL2025! 🐟 We want models that match our [bsky, 63 points, 2 comments]
• arxiv.org/abs/2411.04427 [bsky, 2 points, 0 comments]
• Except it turns out that LLMs don't capture the diversity of actual humans, and tuning the model doesn't help. Which also explains why "creative" work by LLMs all has the same boring sameness. They r [bsky, 0 points, 0 comments]

Related

• Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs
• Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models
• The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models
• Large Language Model Alignment: A Survey
• The myth of language universals: Language diversity and its importance for cognitive science