Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs

2023/11/03 by Qingru Zhang, Zhang, Qingru, Chandan Singh +11 · 7 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2311.02262

openalex publication_date 2023/11/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for the readers to grasp the conveyed information. When interacting with large language models (LLMs), we have a similar need -- steering the model to pay closer attention to user-specified information, e.g., an instruction. Existing methods, however, are constrained to process plain text and do not support such a mechanism. This motivates us to introduce PASTA -- Post-hoc Attention STeering Approach, a method that allows LLMs to read text with user-specified emphasis marks. To this end, PASTA identifies a small subset of attention heads and applies precise attention reweighting on them, directing the model attention to user-specified parts. Like prompting, PASTA is applied at inference time and does not require changing any model parameters. Experiments demonstrate that PASTA can substantially enhance an LLM's ability to follow user instructions or integrate new knowledge from user inputs, leading to a significant performance improvement on a variety of tasks, e.g., an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly available at https://github.com/QingruZhang/PASTA .

Cited by

• Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs
• Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings
• Steering Instruction Hierarchies at Inference Time
• Attention-Aligned Reasoning for Large Language Models
• Enhancing Multi-Agent Communication through Attention Steering with Context Relevance
• Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation
• Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors

Related

• Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering
• Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering
• Attention Instruction: Amplifying Attention in the Middle via Prompting
• Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
• Learning to Look at the Other Side: A Semantic Probing Study of Word Embeddings in LLMs with Enabled Bidirectional Attention