CARE: Decoding Time Safety Alignment via Rollback and Introspection Intervention

2025/09/01 by Xiaoqing Hu, Fei Huang, Hu, Xiaomeng +7 · 1 citation
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Formal Methods in Verification #Machine Learning (cs.LG) #Safety Systems Engineering in Autonomy #Software Reliability and Analysis Research

paper · pdf · doi:10.48550/arxiv.2509.06982

openalex publication_date 2025/09/01 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/28

Abstract

As large language models (LLMs) are increasingly deployed in real-world applications, ensuring the safety of their outputs during decoding has become a critical challenge. However, existing decoding-time interventions, such as Contrastive Decoding, often force a severe trade-off between safety and response quality. In this work, we propose CARE, a novel framework for decoding-time safety alignment that integrates three key components: (1) a guard model for real-time safety monitoring, enabling detection of potentially unsafe content; (2) a rollback mechanism with a token buffer to correct unsafe outputs efficiently at an earlier stage without disrupting the user experience; and (3) a novel introspection-based intervention strategy, where the model generates self-reflective critiques of its previous outputs and incorporates these reflections into the context to guide subsequent decoding steps. The framework achieves a superior safety-quality trade-off by using its guard model for precise interventions, its rollback mechanism for timely corrections, and our novel introspection method for effective self-correction. Experimental results demonstrate that our framework achieves a superior balance of safety, quality, and efficiency, attaining a low harmful response rate and minimal disruption to the user experience while maintaining high response quality.

Citations

• Qwen3Guard Technical Report
• Attention Slipping: A Mechanistic Understanding of Jailbreak Attacks and Defenses in LLMs
• Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
• A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
• Effectively Controlling Reasoning Models through Thinking Intervention
• Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior
• Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
• Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
• Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models
• Qwen2.5 Technical Report
• Jailbreak Attacks and Defenses Against Large Language Models: A Survey
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
• A Framework for Real-time Safeguarding the Text Generation of Large Language Model
• Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
• ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding
• SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
• ARGS: Alignment as Reward-Guided Search
• Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
• Jailbreaking Black Box Large Language Models in Twenty Queries
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
• AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
• Baseline Defenses for Adversarial Attacks Against Aligned Language Models
• Universal and Transferable Adversarial Attacks on Aligned Language Models
• BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
• GPT-4 Technical Report
• Contrastive Decoding: Open-ended Text Generation as Optimization
• In conversation with Artificial Intelligence: aligning language models with human values
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Training language models to follow instructions with human feedback
• A General Language Assistant as a Laboratory for Alignment

Cited by

• Qwen3Guard Technical Report

Related

• SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models
• Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
• DeAL: Decoding-time Alignment for Large Language Models
• The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Models
• Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning