A Diversity-Promoting Objective Function for Neural Conversation Models

2015/10/11 by Jiwei Li, Michel Galley, Li, Jiwei +7 · 167 citations
Computer Science · #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1510.03055

In. Proc of NAACL 2016

arxiv created 2016/06/10 · arxiv updated 2016/06/14

Abstract

Sequence-to-sequence neural network models for generation of conversational responses tend to generate safe, commonplace responses (e.g., "I don't know") regardless of the input. We suggest that the traditional objective function, i.e., the likelihood of output (response) given input (message) is unsuited to response generation tasks. Instead we propose using Maximum Mutual Information (MMI) as the objective function in neural models. Experimental results demonstrate that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations.

Citations

• Attention with Intention for a Neural Network Conversation Model
• Sequence to Sequence Learning with Neural Networks

Cited by

• Do LLM Debates Repeat Arguments Differently Across Languages?
• The One-Word Census: Answer-Choice Conformity Across 44 Language Models
• PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention
• Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS
• CaRE Compute-aware Remasking Evaluation Protocol for Masked Diffusion Language Models
• Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
• Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
• PARAN: Persona-Augmented Review ANswering system on Food Delivery Review Dataset
• Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
• Idea-Gated Transformers: Enforcing Semantic Coherence via Differentiable Vocabulary Pruning
• SciPostLayoutTree: A Dataset for Structural Analysis of Scientific Posters
• HUMORCHAIN: Theory-Guided Multi-Stage Reasoning for Interpretable Multimodal Humor Generation
• Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
• What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity
• ParliaBench: An Evaluation and Benchmarking Framework for LLM-Generated Parliamentary Speech
• Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models
• Revisiting Entropy in Reinforcement Learning for Large Reasoning Models
• Optimizing Diversity and Quality through Base-Aligned Model Collaboration
• Surprisal reveals diversity gaps in image captioning and different scorers change the story
• ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generation
• G2: Guided Generation for Enhanced Output Diversity in LLMs
• Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
• Context-aware Captions from Context-agnostic Supervision
• ReCo: Reweighting GRPO Against Distributional Concentration
• A Hybrid Retrieval-Generation Neural Conversation Model
• Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
• LOGICAL-COMMONSENSEQA: A Benchmark for Logical Commonsense Reasoning
• Efficient semantic uncertainty quantification in language models via diversity-steered sampling
• Correlation Dimension of Auto-Regressive Large Language Models
• A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness
• Speculative Model Risk in Healthcare AI: Using Storytelling to Surface Unintended Harms
• ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding
• FreshTab: Sourcing Fresh Data for Table-to-Text Generation Evaluation
• Beating Harmful Stereotypes Through Facts: RAG-based Counter-speech Generation
• SAGE: A Top-Down Bottom-Up Knowledge-Grounded User Simulator for Multi-turn AGent Evaluation
• Towards Self-Refinement of Vision-Language Models with Triangular Consistency
• Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
• RoSE: Round-robin Synthetic Data Evaluation for Selecting LLM Generators without Human Test Sets
• NonTextual Target Attack
• An Ensemble Dialogue System for Facts-Based Sentence Generation
• Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning
• MGen: Millions of Naturally Occurring Generics in Context
• Distillation of Large Language Models via Concrete Score Matching
• CARE: Cognitive-reasoning Augmented Reinforcement for Emotional Support Conversation
• SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
• Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems
• Following the TRACE: A Structured Path to Empathetic Response Generation with Multi-Agent Models
• AutoSpec: An Agentic Framework for Automatically Drafting Patent Specification
• Gunrock 2.0: A User Adaptive Social Conversational System
• CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models
• Weights-Rotated Preference Optimization for Large Language Models
• Asking a Language Model for Diverse Responses
• Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
• Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization
• Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
• Empathy-R1: A Chain-of-Empathy and Reinforcement Learning Framework for Long-Form Mental Health Support
• Personalized Dialogue Generation with Diversified Traits
• Chat-of-Thought: Collaborative Multi-Agent System for Generating Domain Specific Information
• CoachMe: Decoding Sport Elements with a Reference-Based Coaching Instruction Generation Model
• Pluralistic Off-policy Evaluation and Alignment
• GRAID: Synthetic Data Generation with Geometric Constraints and Multi-Agentic Reflection for Harmful Content Detection
• We Argue to Agree: Towards Personality-Driven Argumentation-Based Negotiation Dialogue Systems for Tourism
• Differentially-private text generation degrades output language quality
• Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling
• Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Prompting
• MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
• AI-in-the-Loop: Privacy Preserving Real-Time Scam Detection and Conversational Scambaiting by Leveraging LLMs and Federated Learning
• ProMQA-Assembly: Multimodal Procedural QA Dataset on Assembly
• Jointly Reinforcing Diversity and Quality in Language Model Generations
• Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation
• Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs
• Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning
• The Ramon Llull's Thinking Machine for Automated Ideation
• Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models
• Beyond Benchmark: LLMs Evaluation with an Anthropomorphic and Value-oriented Roadmap
• Adaptive Originality Filtering: Rejection Based Prompting and RiddleScore for Culturally Grounded Multilingual Riddle Generation
• Transplant Then Regenerate: A New Paradigm for Text Data Augmentation
• Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses
• Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation
• E3RG: Building Explicit Emotion-driven Empathetic Response Generation System with Multimodal Large Language Model
• SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance
• Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimization
• Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores
• Which Kind Is Better in Open-domain Multi-turn Dialog,Hierarchical or Non-hierarchical Models? An Empirical Study
• Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation
• FIRE: Faithful Interpretable Recommendation Explanations
• Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
• DTPA: Dynamic Token-level Prefix Augmentation for Controllable Text Generation
• RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior
• Can LLMs Generate High-Quality Task-Specific Conversations?
• Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training
• A survey of diversity quantification in natural language processing: The why, what, where and how
• Diversity-Enhanced Reasoning for Subjective Questions
• Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders
• DRIVE: Disfluency-Rich Synthetic Dialog Data Generation Framework for Intelligent Vehicle Environments
• E.A.R.T.H.: Structuring Creative Evolution through Model Error in Generative AI
• Turn-level Dialog Evaluation with Dialog-level Weak Signals for Bot-Human Hybrid Customer Service Systems
• Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Models
• GEMMAS: Graph-based Evaluation Metrics for Multi Agent Systems
• Why Do Neural Response Generation Models Prefer Universal Replies?
• Toward Real-World Chinese Psychological Support Dialogues: CPsDD Dataset and a Co-Evolving Multi-Agent System
• Reward Models Can Improve Themselves: Reward-Guided Adversarial Failure Mode Discovery for Robust Reward Modeling
• A Survey of Pun Generation: Datasets, Evaluations and Methodologies
• Alignment Is Local: A Paired Diagnostic for GUI Agents under User-Side Persuasion
• Predicting User Engagement Status for Online Evaluation of Intelligent Assistants
• Towards the Holographic Characteristic of LLMs for Efficient Short-text Generation
• Relevance-Promoting Language Model for Short-Text Conversation
• Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder
• PromptAug: Fine-grained Conflict Classification Using Data Augmentation
• What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning
• RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models
• Statistical Multicriteria Evaluation of LLM-Generated Text
• LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
• SocialSim: Towards Socialized Simulation of Emotional Support Conversation
• GeoGuess: Multimodal Reasoning based on Hierarchy of Visual Information in Street View
• Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability
• Modeling the One-to-Many Property in Open-Domain Dialogue with LLMs
• Value-Free Policy Optimization via Reward Partitioning
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction
• From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation
• No Data? No Problem: Synthesizing Security Graphs for Better Intrusion Detection
• STORYTELLER: An Enhanced Plot-Planning Framework for Coherent and Cohesive Story Generation
• When LLMs Team Up: The Emergence of Collaborative Affective Computing
• KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors
• FACE: A Fine-grained Reference Free Evaluator for Conversational Recommender Systems
• Automatic, Personalized, and Flexible Playlist Generation using Reinforcement Learning
• Beyond Multiple Choice: Evaluating Steering Vectors for Summarization
• From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning
• Can LLMs Deceive CLIP? Benchmarking Adversarial Compositionality of Pre-trained Multimodal Representation via Text Updates
• From Motion to Behavior: Hierarchical Modeling of Humanoid Generative Behavior Control
• BiasFilter: An Inference-Time Debiasing Framework for Large Language Models
• Learning Natural Language Generation from Scratch
• Silencer: From Discovery to Mitigation of Self-Bias in LLM-as-Benchmark-Generator
• OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions
• Creativity in LLM-based Multi-Agent Systems: A Survey
• Preference Optimization by Estimating the Ratio of the Data Distribution
• Latent Intention Dialogue Models
• BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling
• Diverse, not Short: A Length-Controlled Data Selection Strategy for Improving Response Diversity of Language Models
• DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimization
• From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment
• Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation
• Text-to-Pipeline: Bridging Natural Language and Data Preparation Pipelines
• Emotional Supporters often Use Multiple Strategies in a Single Turn
• The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support
• Visual Instruction Bottleneck Tuning
• Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis
• Shaping the Narrative Arc: An Information-Theoretic Approach to\n Collaborative Dialogue
• ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
• Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models
• Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidance
• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics
• Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward
• Generative Deep Neural Networks for Dialogue: A Short Review
• TF1-EN-3M: Three Million Synthetic Moral Fables for Training Small, Open Language Models
• Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decoding
• Controlling Repetition in Protein Language Models
• Low-Resource Knowledge-Grounded Dialogue Generation
• On the Diversity of Analogy Making in Large Language Models
• Robo-Troj: Attacking LLM-based Task Planners
• Tree of Thoughts as a Classical Heuristic Search Problem: Formal Foundations and Design Patterns
• NoveltyBench: Evaluating Language Models for Humanlike Diversity
• Chat More If You Like: Dynamic Cue Words Planning to Flow Longer Conversations
• Evaluating the Diversity and Quality of LLM Generated Content
• DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
• A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis
• From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM

Related

• Why Do Neural Response Generation Models Prefer Universal Replies?
• Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
• Jointly Optimizing Diversity and Relevance in Neural Response Generation
• Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
• Another Diversity-Promoting Objective Function for Neural Dialogue Generation