In-Context Learning Creates Task Vectors

2023/10/24 by Roee Hendel, Mor Geva, Hendel, Roee +3 · 3 voices · 97 citations
Computer Science · Decision Sciences · #Computation and Language (cs.CL) #Data Quality and Management #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2310.15916

openalex publication_date 2023/10/24 · arxiv published 2023/10/24 · arxiv updated 2023/10/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In particular, it is challenging to map it to the "standard" machine learning framework, where one uses a training set S to find a best-fitting function f(x) in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query x and a single "task vector" calculated from the training set. Thus, ICL can be seen as compressing S into a single task vector \boldsymbolθ(S) and then using this task vector to modulate the transformer to produce the output. We support the above claim via comprehensive experiments across a range of models and tasks.

Cited by

• Decomposing Task Vectors for Refined Model Editing
• Distributed Sparse Interventions in Language Models
• Where Steering Signals Come From: Activation Source Selection in Activation Steering
• In-Context Algebra
• Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations
• The Curious Case of Analogies: Investigating Analogical Reasoning in Large Language Models
• Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning
• Beyond Plain Demos: A Demo-centric Anchoring Paradigm for In-Context Learning in Alzheimer's Disease Detection
• Implicit Federated In-context Learning For Task-Specific LLM Fine-Tuning
• You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
• In-Context Learning Without Copying
• Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models
• Think Outside the Policy: In-Context Steered Policy Optimization
• France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions
• How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs
• Context Structure Reshapes the Representational Geometry of Language Models
• Sequences of Logits Reveal the Low Rank Structure of Language Models
• Vocab Diet: Reshaping the Vocabulary of LLMs with Vector Arithmetic
• Multimodal Function Vectors for Visual Relations
• Exploring and Leveraging Class Vectors for Classifier Editing
• On the Representations of Entities in Auto-regressive Large Language Models
• Memory Retrieval and Consolidation in Large Language Models through Function Tokens
• Transmuting prompts into weights
• Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?
• TDHook: A Lightweight Framework for Interpretability
• Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
• Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
• Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions
• Train Once, Reuse Everywhere: Generalizable Implicit In-Context Learning by Routing Attention
• A circuit for predicting hierarchical structure in-context in Large Language Models
• Binary Autoencoder for Mechanistic Interpretability of Large Language Models
• Stability of In-Context Learning: A Spectral Coverage Perspective
• Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
• Variational Task Vector Composition
• DISCO: Disentangled Communication Steering for Large Language Models
• All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens
• Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
• Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
• Just-in-time and distributed task representations in language models
• Uncovering Spontaneous Physics Representations in In-Context Learning
• ICM-Fusion: In-Context Meta-Optimized LoRA Fusion for Multi-Task Adaptation
• When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models
• Lucy: edgerunning agentic web search on mobile with machine generated task vectors
• Enhancing Project-Specific Code Completion by Inferring Internal API Information
• SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
• Towards Compute-Optimal Many-Shot In-Context Learning
• Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
• Function Induction and Task Generalization: An Interpretability Study with Off-by-One Addition
• Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
• Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations
• Scaling can lead to compositional generalization
• Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
• Activation Reward Models for Few-Shot Model Alignment
• Latent Concept Disentanglement in Transformer-based Language Models
• Next-Token Prediction Should be Ambiguity-Sensitive: A Meta-Learning Perspective
• InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
• Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs
• Distinct Computations Emerge From Compositional Curricula in In-Context Learning
• Assessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relation
• Surprise Calibration for Better In-Context Learning
• Understanding In-context Learning of Addition via Activation Subspaces
• Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering
• Contextually Guided Transformers via Low-Rank Adaptation
• Adaptive Task Vectors for Large Language Models
• Beyond Multiple Choice: Evaluating Steering Vectors for Summarization
• One Task Vector is not Enough: A Large-Scale Study for In-Context Learning
• Understanding (Un)Reliability of Steering Vectors in Language Models
• Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
• Evaluating Steering Techniques using Human Similarity Judgments
• Optimization-Inspired Few-Shot Adaptation for Large Language Models
• Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning
• The emergence of sparse attention: impact of data distribution and benefits of repetition
• Beyond Demonstrations: Dynamic Vector Construction from Latent Representations
• From Compression to Expression: A Layerwise Analysis of In-Context Learning
• Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
• The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation
• Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs
• Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations
• Do different prompting methods yield a common task representation in language models?
• Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
• Large Language Models provide support for the parallelogram theory of analogy
• In-Context Collapse in Vision-Language Models and How to Mitigate it?
• Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning
• Large language models reorganize representational geometry during in-context learning
• Improving Reasoning Performance in Large Language Models via Representation Engineering
• What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?
• The Information Geometry of Softmax: Probing and Steering
• Language Models Struggle to Use Representations Learned In-Context
• DeepInsert: Early Layer Bypass for Efficient and Performant Multimodal Understanding
• Patches of Nonlinearity: Instruction Vectors in Large Language Models
• What do Language Models Learn and When? The Implicit Curriculum Hypothesis
• Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages
• Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims
• Steering off Course: Reliability Challenges in Steering Language Models
• M²IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineering
• Functional Abstraction of Knowledge Recall in Large Language Models
• On Linear Representations and Pretraining Data Frequency in Language Models
• KV-Skill: Forging Expertise in the Model's Native Language
• When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
• Mimic In-Context Learning for Multimodal Tasks

Discussions

• Oops working link arxiv.org/abs/2310.15916 [bsky, 3 points, 0 comments]
• In-Context Learning Creates Task Vectors [hn, 3 points, 0 comments]
• Researchers discover that in-context learning creates task vectors in LLMs arxiv.org/pdf/2310.159... [bsky, 0 points, 0 comments]

Related

• Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
• Label Words as Local Task Vectors in In-Context Learning
• Learning Task Representations from In-Context Learning
• One Task Vector is not Enough: A Large-Scale Study for In-Context Learning
• What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning