2025/07/06 by Jiangbo Lu, Jack Lu, Ryan Teehan +6 · 2 voices · 2 citations
Computer Science · Engineering · Medicine · #Building Energy and Comfort Optimization #Context-Aware Activity Recognition Systems #Healthcare Technology and Patient Monitoring #cs.AI #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.2507.04221
openalex publication_date 2025/07/06 · openalex created_date 2025/10/20 · openalex updated_date 2026/07/28
We introduce Context Tuning, a simple and effective method to significantly enhance few-shot adaptation of large language models (LLMs) without weight updates. In-Context Learning (ICL) forms a memory representation of the demonstrations in a single forward pass but cannot refine it when insufficient. Prompt-based methods offer lightweight adaptation by optimizing a trainable prompt or prefix but initialize it independently of the demonstrations. In contrast, Context Tuning leverages the model's inherent ICL ability to initialize a trainable memory representation from demonstrations, then refines it through gradient-based optimization. Extensive evaluations on benchmarks such as CrossFit, UnifiedQA, MMLU, BIG-Bench Hard, and ARC demonstrate that Context Tuning outperforms both ICL and traditional prompt-based adaptation methods while achieving competitive accuracy with Test-Time Training at significantly higher training efficiency.