A Survey on LLM Inference-Time Self-Improvement

2024/12/18 by Xiangjue Dong, Dong, Xiangjue, Maria Teleki +3 · 5 citations
Computer Science · #Advanced Computational Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2412.14352

Abstract

Techniques that enhance inference through increased computation at test-time have recently gained attention. In this survey, we investigate the current state of LLM Inference-Time Self-Improvement from three different perspectives: Independent Self-improvement, focusing on enhancements via decoding or sampling methods; Context-Aware Self-Improvement, leveraging additional context or datastore; and Model-Aided Self-Improvement, achieving improvement through model collaboration. We provide a comprehensive review of recent relevant studies, contribute an in-depth taxonomy, and discuss challenges and limitations, offering insights for future research.

Cited by

• A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solving
• SDA: Steering-Driven Distribution Alignment for Open LLMs without Fine-Tuning
• Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention
• Self-Improvement in Multimodal Large Language Models: A Survey
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally

Related

• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
• A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
• Review of Inference-Time Scaling Strategies: Reasoning, Search and RAG
• Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens