Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs
2025/12/03 by Ting Ma, Jiaqi Yao, Ma, Tengyun +10
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Security and Verification in Computing #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2512.03720
openalex publication_date 2025/12/03 · openalex created_date 2025/12/05 · openalex updated_date 2026/07/28
Abstract
Large Language Models (LLMs) have emerged as powerful tools for diverse applications. However, their uniform token processing paradigm introduces critical vulnerabilities in instruction handling, particularly when exposed to adversarial scenarios. In this work, we identify and propose a novel class of vulnerabilities, termed Tool-Completion Attack (TCA), which exploits function-calling mechanisms to subvert model behavior. To evaluate LLM robustness against such threats, we introduce the Tool-Completion benchmark, a comprehensive security assessment framework, which reveals that even state-of-the-art models remain susceptible to TCA, with surprisingly high attack success rates. To address these vulnerabilities, we introduce Context-Aware Hierarchical Learning (CAHL), a sophisticated mechanism that dynamically balances semantic comprehension with role-specific instruction constraints. CAHL leverages the contextual correlations between different instruction segments to establish a robust, context-aware instruction hierarchy. Extensive experiments demonstrate that CAHL significantly enhances LLM robustness against both conventional attacks and the proposed TCA, exhibiting strong generalization capabilities in zero-shot evaluations while still preserving model performance on generic tasks. Our code is available at https://github.com/S2AILab/CAHL.
Citations
- Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering
- The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)
- Defense against Prompt Injection Attacks via Mixture of Encodings
- ASIDE: Architectural Separation of Instructions and Data in Language Models
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
- Qwen2.5 Technical Report
- A Survey on LLM-as-a-Judge
- Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning
- Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy
- SecAlign: Defending Against Prompt Injection with Preference Optimization
- Improving Alignment and Robustness with Circuit Breakers
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- LLoCO: Learning Long Contexts Offline
- Long-Context Language Modeling with Parallel Context Encoding
- StruQ: Defending Against Prompt Injection with Structured Queries
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
- Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- Representation Engineering: A Top-Down Approach to AI Transparency
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- Effective Prompt Extraction from Language Models
- Prompt Injection attack against LLM-integrated Applications
- AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- Toolformer: Language Models Can Teach Themselves to Use Tools
- Ignore Previous Prompt: Attack Techniques For Language Models
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- Emergent Abilities of Large Language Models
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Training language models to follow instructions with human feedback
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- 8-bit Optimizers via Block-wise Quantization
- Reframing Instructional Prompts to GPTk's Language
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Evaluating Large Language Models Trained on Code
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- Extracting Training Data from Large Language Models
- Attention Is All You Need
- AdvAgent: Controllable Blackbox Red-teaming on Web Agents
Related