Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

2025/10/01 by Jiayi Zhang, Simon Yu, Simon C.H. Yu +12 · 14 voices · 12 citations
Immunology and Microbiology · #Biosimilars and Bioanalytical Methods #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2510.01171

openalex publication_date 2025/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically favor familiar text as a result of well-established findings in cognitive psychology. We formalize this bias theoretically, verify it on preference datasets empirically, and show that it plays a central role in mode collapse. Motivated by this analysis, we introduce Verbalized Sampling, a simple, training-free prompting strategy to circumvent mode collapse. VS prompts the model to verbalize a probability distribution over a set of responses (e.g., "Generate 5 jokes about coffee and their corresponding probabilities"). Comprehensive experiments show that VS significantly improves performance across creative writing (poems, stories, jokes), dialogue simulation, open-ended QA, and synthetic data generation, without sacrificing factual accuracy and safety. For instance, in creative writing, VS increases diversity by 1.6-2.1x over direct prompting. We further observe an emergent trend that more capable models benefit more from VS. In sum, our work provides a new data-centric perspective on mode collapse and a practical inference-time remedy that helps unlock pre-trained generative diversity.

Cited by

• Systematic Literature Reviews With Two Multi-Agentic Systems And Human-In-The-Loop
• Simulating Human Memory with Language Models
• More Is Not More: What Matters for Diversity in LLM Opinions?
• Discourse Diversity in Multi-Turn Empathic Dialogue
• MAR:Multi-Agent Reflexion Improves Reasoning Abilities in LLMs
• QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models
• Verbalizing LLMs' assumptions to explain and control sycophancy
• Cognitive Alpha Mining via LLM-Driven Code-Based Evolution
• Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution
• Reasoning About Intent for Ambiguous Requests
• CREATE: Testing LLMs for Associative Creativity
• Argument Collapse: LLMs Flatten Long-Form Public Debate

Discussions

• *Your AI only SEEMS like a boring parrot because of its mode-collapse typicality-bias arxiv.org/pdf/2510.01171 [bsky, 10 points, 0 comments]
• Maybe verbalized sampling? arxiv.org/abs/2510.01171 [bsky, 5 points, 0 comments]
• Here is the Stanford paper on better prompting arxiv.org/pdf/2510.01171 [bsky, 3 points, 0 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity [hn, 3 points, 0 comments]
• Verbalized Sampling: Diversity is just hidden. 📄Paper: arxiv.org/abs/2510.01171 🌐Blog: verbalized-sampling.com Team: Jiayi Zhang @simon-ycl.bsky.social @derekch.bsky.social Anthony Sicilia, Michael [bsky, 3 points, 0 comments]
• 急に流れてきたので。 arxiv.org/abs/2510.01171 [bsky, 2 points, 0 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity [hn, 2 points, 0 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity [hn, 1 points, 2 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity arxiv.org/abs/2510.01171 [bsky, 1 points, 0 comments]
• Helpful way of improving LLM output: Verbalized Sampling arxiv.org/pdf/2510.01171 "generate a set of N possible responses, each within a separate <response> tag. Responses should each include a <text> [bsky, 1 points, 0 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock #LLM Diversity arxiv.org/abs/2510.01171 [bsky, 1 points, 0 comments]
• Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity [hn, 1 points, 0 comments]
• >> arxiv.org/abs/2510.01171 <=--~--~- [bsky, 0 points, 0 comments]
• The paper in question is Zhang, et al on “Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity” arxiv.org/abs/2510.01171 [bsky, 0 points, 0 comments]

Related

• Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models
• The African Woman is Rhythmic and Soulful: An Investigation of Implicit Biases in LLM Open-ended Text Generation
• Diverse, not Short: A Length-Controlled Data Selection Strategy for Improving Response Diversity of Language Models
• Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs
• HyPerAlign: Interpretable Personalized LLM Alignment via Hypothesis Generation