The Surprising Effectiveness of Test-Time Training for Few-Shot Learning

2024/11/11 by Ekin Akyürek, Akyürek, Ekin, Mehul Damani +14 · 17 voices · 40 citations
Computer Science · #Intelligent Tutoring Systems and Adaptive Learning

paper · pdf · doi:10.48550/arxiv.2411.07279

Abstract

Language models (LMs) have shown impressive performance on tasks within their training distribution, but often struggle with structurally novel tasks even when given a small number of in-context task examples. We investigate the effectiveness of test-time training (TTT) -- temporarily updating model parameters during inference using a loss derived from input data -- as a mechanism for improving LMs' reasoning and few-shot learning capabilities. On the Abstraction and Reasoning Corpus (ARC), performing TTT with in-context examples yields up to 6× higher accuracy compared to fine-tuned baselines -- reaching 53.0% on the public validation set with an 8B-parameter LM and 61.9% when ensembled with program-synthesis methods, matching average human performance. On BIG-Bench Hard (BBH), TTT on in-context examples surpasses standard few-shot prompting in the 10-shot setting by 7.3 percentage points (50.5% to 57.8%). Our findings highlight the limitations of in-context learning for novel tasks and demonstrate the potential of test-time training to enhance language model adaptability.

Cited by

• Memoir: Should a Model Write to Its Memory While It Thinks?
• Self-Adapting Language Models
• OptPO: Optimal Rollout Allocation for Test-time Policy Optimization
• Reinforcement Learning via Self-Distillation
• Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training
• Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
• ARC-AGI Without Pretraining
• Llama-based source code vulnerability detection: Prompt engineering vs Fine tuning
• Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning
• SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization
• Think Visually, Reason Textually: Vision-Language Synergy in ARC
• Test-Time Adaptation for LLM Agents via Environment Interaction
• ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus
• Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
• Multi-Stakeholder Alignment in LLM-Powered Collaborative AI Systems: A Multi-Agent Framework for Intelligent Tutoring
• Shoot First, Ask Questions Later? Building Rational Agents that Explore and Act Like People
• Finding the Sweet Spot: Trading Quality, Cost, and Speed During Inference-Time LLM Reflection
• Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs
• Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
• You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
• SIA: Self Improving AI with Harness & Weight Updates
• Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models
• Self-Improving LLM Agents at Test-Time
• End-to-End Test-Time Training for Long Context
• Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
• Efficient Test-Time Scaling for Small Vision-Language Models
• Aristotle: IMO-level Automated Theorem Proving
• Test time training enhances in-context learning of nonlinear functions
• LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
• Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
• ReasonCACHE: Teaching LLMs To Reason Without Weight Updates
• Test-Time Warmup for Multimodal Large Language Models
• ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory
• CausalARC: Abstract Reasoning with Causal World Models
• MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time
• RTTC: Reward-Guided Collaborative Test-Time Compute
• Channel-Wise MLPs Improve the Generalization of Recurrent Convolutional Networks
• LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems
• Test-time Offline Reinforcement Learning on Goal-related Experience
• Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning

Discussions

• One thing still missing here is good discussion of academic papers, so I guess I'll be the change I want to see in the world Really interesting results from Jacob Andreas's group showing great perfor [bsky, 182 points, 7 comments]
• It is certainly looking like larger and larger LLMs won't get to AGI. Was reading an article on @theinformation.bsky.social today about how companies are looking at other strategies (arxiv.org/abs/241 [bsky, 7 points, 1 comments]
• With a combination of new test-time compute techniques and smart scaffolding, we might crack the ARC benchmark in the next 6 months. What will be left in few years time? arxiv.org/abs/2411.07279 x.c [bsky, 7 points, 0 comments]
• AI Matches Human Reasoning Without Symbolic Logic MIT research reveals AI achieves human-level performance on abstract reasoning tasks through test-time training, reaching 61.9% accuracy. https://arx [bsky, 6 points, 0 comments]
• The Surprising Effectiveness of Test-Time Training for Few-Shot Learning [hn, 3 points, 0 comments]
• *The Surprising Effectiveness of Test-Time Training for Abstract Reasoning* by @ekinakyurek.bsky.social et al. Shows that test-time training (fine-tuning at inference time) strongly improves performa [bsky, 3 points, 0 comments]
• MIT Pushes AI to Human-Level Reasoning MIT’s Test-Time Training (TTT) boosts LLMs' abstract reasoning, achieving human-level performance on the ARC benchmark. A breakthrough in AI's adaptability and [bsky, 3 points, 0 comments]
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning [hn, 3 points, 0 comments]
• So 'test-time training' = making the model do statistical predictions about its statistical predictions? Sure! Why not! arxiv.org/abs/2411.07279 [bsky, 2 points, 1 comments]
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning [hn, 1 points, 0 comments]
• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning [hn, 1 points, 0 comments]
• TTT propels AI to rival human reasoning with unprecedented accuracy. This is more than progres it’s a shift in the balance of intelligence. Are we truly prepared for when machines outthink us in ways [bsky, 1 points, 1 comments]
• New research from #MIT suggests explicit symbolic search isn’t the only route to enhancing abstract reasoning in LLMs. Test-time training #TTT— temporarily updating model parameters during inference—c [bsky, 1 points, 0 comments]
• Not quite lol. arxiv.org/abs/2411.07279 [bsky, 1 points, 0 comments]
• Finally catching up on this Test Time Training paper. Thoughts - very nice results - interesting that TTT helps smaller models more (sign that scale is disproportionately useful for more memorizati [bsky, 1 points, 1 comments]
• “The Surprising Effectiveness of Test-Time Training for Abstract Reasoning” arxiv.org/abs/2411.07279 [bsky, 0 points, 0 comments]
• interesting paper arxiv.org/abs/2411.07279 a method that temporarily updates model parameters during inference. combined with specific techniques like initial finetuning and auxiliary tasks, significa [bsky, 0 points, 0 comments]

Related

• Fine-Tuned In-Context Learners for Efficient Adaptation
• True Few-Shot Learning with Language Models
• Making Pre-trained Language Models Better Few-shot Learners
• Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
• Test-Time Learning for Large Language Models