Token-Budget-Aware LLM Reasoning

2024/12/24 by Tingxu Han, Han, Tingxu, Zhenting Wang +9 · 38 citations
Computer Science · #Artificial Intelligence (cs.AI) #Blockchain Technology Applications and Security #Computation and Language (cs.CL) #Cryptography and Data Security #Digital Rights Management and Security #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2412.18547

openalex publication_date 2024/12/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reasoning is critical for large language models (LLMs) to excel in a wide range of tasks. While methods like Chain-of-Thought (CoT) reasoning and enhance LLM performance by decomposing problems into intermediate steps, they also incur significant overhead in token usage, leading to increased costs. We find that the reasoning process of current LLMs is unnecessarily lengthy and it can be compressed by including a reasonable token budget in the prompt, but the choice of token budget plays a crucial role in the actual compression effectiveness. We then propose a token-budget-aware LLM reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. Experiments show that our method effectively reduces token costs in CoT reasoning with only a slight performance reduction, offering a practical solution to balance efficiency and accuracy in LLM reasoning. Code: https://github.com/GeniusHTX/TALE

Cited by

• Specification and Detection of LLM Code Smells
• FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaboration
• Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases
• Plantain: Plan-Answer Interleaved Reasoning
• Budget-Aware Tool-Use Enables Effective Agent Scaling
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
• Investigating CoT Monitorability in Large Reasoning Models
• Logit-Entropy Adaptive Stopping Heuristic for Efficient Chain-of-Thought Reasoning
• Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs
• DeepCompress: A Dual Reward Strategy for Dynamically Exploring and Compressing Reasoning Chains
• Are Language Models Efficient Reasoners? A Perspective from Logic Programming
• ProofSketch: Efficient Verified Reasoning for Large Language Models
• EdgeReasoning: Characterizing Reasoning LLM Deployment on Edge GPUs
• Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization
• Are Large Reasoning Models Interruptible?
• Merlin's Whisper: Enabling Efficient Reasoning in LLMs via Black-box Adversarial Prompting
• One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
• Concise Reasoning in the Lens of Lagrangian Optimization
• Proof Strategy Extraction from LLMs for Enhancing Symbolic Provers
• Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning
• Mitigating Overthinking through Reasoning Shaping
• The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
• MixReasoning: Switching Modes to Think
• Improving Chain-of-Thought Efficiency for Autoregressive Image Generation
• DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
• ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models
• Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement Learning
• Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughts
• Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility
• Early Stopping Chain-of-thoughts in Large Language Models
• BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
• From Long to Short: LLMs Excel at Trimming Own Reasoning Chains
• Self-Aligned Reward: Towards Effective and Efficient Reasoners
• MeVe: A Modular System for Memory Verification and Effective Context Control in Language Models
• Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoning
• Efficient Agents: Building Effective Agents While Reducing Cost

Related

• Steering LLM Thinking with Budget Guidance
• TokenSkip: Controllable Chain-of-Thought Compression in LLMs
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
• BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
• SABER: Switchable and Balanced Training for Efficient LLM Reasoning