Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations

2026/04/29 by Mingqian Zheng, Malia Morgan, Liwei Jiang +2 · 1 voice
Computer Science · #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2604.27093

Abstract

Current LLM safety alignment techniques improve model robustness against adversarial attacks, but overlook whether and how LLMs can recover helpfulness when benign users clarify their intent. We introduce CarryOnBench, the first interactive benchmark that measures whether LLMs can revise their interpretation of user intent and recover utility, while remaining safe through multi-turn conversations. Starting from 398 seemingly harmful queries with benign underlying intents, we simulate 5,970 conversations by varying user follow-up sequences, evaluating 14 models on both intent-aligned utility and safety. CarryOnBench yields 1,866 different conversation flows of 4--12 turns, totaling 23,880 model responses. We design Ben-Util, a checklist-based metric that evaluates how well each model response fulfills the user's benign information need using atomic items. At turn one, models fulfill only 10.5--37.6% of the user's benign information need. When the same query includes the benign intent upfront, models fulfill 25.1--72.1%, confirming that models withhold information due to intent misinterpretation, not limited knowledge. With benign clarifications in multi-turn conversations, 13 of 14 models approach or exceed this single-turn baseline, yet recovery cost varies across models. We identify three failure modes invisible to single-turn evaluations: utility lock-in, where a model rarely updates despite clarification; unsafe recovery, where a model updates at disproportionate safety cost; and repetitive recovery, where a model recycles prior responses rather than providing new information. Moreover, conversations converge to similar harmfulness levels regardless of how conservative the model starts. These findings expose a gap that single-turn evaluations miss -- whether a model is appropriately cautious or simply unresponsive to clarified user intent.

Citations

• Mind the Sim2Real Gap in User Simulation for Agentic Tasks
• Understanding and Mitigating Over-refusal for Large Language Models via Safety Representation
• Accumulating Context Changes the Beliefs of Language Models
• AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
• Dynamic Evaluation for Oversensitivity in LLMs
• OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
• Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
• Reinforcement Learning with Rubric Anchors
• Checklists Are Better Than Reward Models For Aligning Language Models
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
• DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
• Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences
• Words Like Knives: Backstory-Personalized Modeling and Detection of Violent Communication
• What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts
• FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning
• LLMs Get Lost In Multi-Turn Conversation
• X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
• Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics
• StructFlowBench: A Structured Flow Benchmark for Multi-turn Instruction Following
• Prioritization First, Principles Second: An Adaptive Interpretation of Helpful, Honest, and Harmless Principles
• CollabLLM: From Passive Responders to Active Collaborators
• CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
• A Survey on Multi-Turn Interaction Capabilities of Large Language Models
• TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation
• Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models
• The Art of Saying No: Contextual Noncompliance in Language Models
• WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
• OR-Bench: An Over-Refusal Benchmark for Large Language Models
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training
• Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack
• A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems
• Are LLM-based Evaluators Confusing NLG Quality Criteria?
• Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs
• Do LLMs exhibit human-like response biases? A case study in survey design
• Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces
• Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
• XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements
• A Survey of Evaluation Metrics Used for NLG Systems

Discussions

• The safety-utility tradeoff isn't a fixed property of models. It's largely unresolved ambiguity that multi-turn interaction can resolve. The question isn't whether a model refuses — it's whether it ca [bsky, 1 points, 1 comments]

Related

• Robust Critics: Defending LLMs Against Multi-Turn Attacks
• One More Turn, Less Regret: A Regret-Based Multi-Turn Benchmark for LLMs' Clarification Policies
• Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue
• IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement
• SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals