Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

2025/06/26 by Jiashuo Wang, Wang, Jiashuo, Kaitao Song +13
Computer Science · Social Sciences · #AI in Service Interactions #Computation and Language (cs.CL) #FOS: Computer and information sciences #Knowledge Management and Sharing #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2506.21497

openalex publication_date 2025/06/26 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28

Abstract

Enhancing user engagement through interactions plays an essential role in socially-driven dialogues. While prior works have optimized models to reason over relevant knowledge or plan a dialogue act flow, the relationship between user engagement and knowledge or dialogue acts is subtle and does not guarantee user engagement in socially-driven dialogues. To this end, we enable interactive LLMs to learn user engagement by leveraging signals from the future development of conversations. Specifically, we adopt a more direct and relevant indicator of user engagement, i.e., the user's reaction related to dialogue intention after the interaction, as a reward to align interactive LLMs. To achieve this, we develop a user simulator to interact with target interactive LLMs and explore interactions between the user and the interactive LLM system via i×MCTS (Monte Carlo Tree Search for interaction). In this way, we collect a dataset containing pairs of higher and lower-quality experiences using i×MCTS, and align interactive LLMs for high-level user engagement by direct preference optimization (DPO) accordingly. Experiments conducted on two socially-driven dialogue scenarios (emotional support conversations and persuasion for good) demonstrate that our method effectively enhances user engagement in interactive LLMs.

Citations

• PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling
• Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning
• The Llama 3 Herd of Models
• ComPeer: A Generative Conversational Agent for Proactive Peer Support
• Towards a Client-Centered Assessment of LLM Therapists by Client Simulation
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
• LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
• SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations
• Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
• Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning
• Knowledge-enhanced Mixed-initiative Dialogue System for Emotional Support Conversations
• CARE: Causality Reasoning for Empathetic Responses by Conditional Graph Generation
• Mastering Atari Games with Limited Data
• PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support
• Towards Emotional Support Dialog Systems
• Gunrock: A Social Bot for Complex and Engaging Long Conversations
• Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good
• Learning Conversational Systems that Interleave Task and Non-Task Content
• School Engagement: Potential of the Concept, State of the Evidence
• ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents

Related

• Aligning LLMs with Individual Preferences via Interaction
• CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions
• Interaction Dynamics as a Reward Signal for LLMs
• Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager
• Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generation