Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning

2026/04/07 by Qisheng Su, Shiting Huang, Zhen Fang +3 · 1 voice
Computer Science · #AI-based Problem Solving and Planning #Advanced Software Engineering Methodologies #Constraint Satisfaction and Optimization #Context (archaeology) #Forcing (mathematics) #Inefficiency #Inference #Metric (unit) #Quality (philosophy) #Security token #cs.PF #cs.SE

paper · pdf · doi:10.48550/arxiv.2604.05404

openalex publication_date 2026/04/07 · arxiv published 2026/04/07 · openalex created_date 2026/04/09 · arxiv updated 2026/04/14 · openalex updated_date 2026/07/28

Abstract

In real-world Tool-Integrated Reasoning (TIR) scenarios, where LLMs interleave reasoning with external tool calls, a major source of inefficiency is that the toolcalls create pauses between LLM requests and cause KV-Cache eviction, forcing recomputation. Also, the long, unfiltered response returned by external tools inflates the KV-Cache, so each decode step spends more time loading the growing cache and thus becomes steadily slower as context length increases. However, existing efficiency metrics like token counts and toolcall counts fail to capture the real model inference latency. To address this, we introduce PTE (Prefill Token Equivalents), a hardware-aware TIR-efficiency metric that unifies internal reasoning and external tool-use costs while explicitly accounting for non-reusable KV-Cache and long-tool-response scenarios. Validation in a high-concurrency industrial setting indicates that PTE aligns significantly better with wall-clock latency than standard token counts, while maintaining consistent efficiency rankings across diverse hardware profiles. We conduct extensive experiments across five TIR benchmarks, quantify their PTE costs, and identify four inefficiency patterns that appear in TIR. We also discover that trajectories with higher PTE costs tend to have lower reasoning correctness, indicating that simply using more tools does not improve the quality of the answer.

Citations

• CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents
• Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
• Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values
• How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
• R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?
• Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning
• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning
• Efficient Agents: Building Effective Agents While Reducing Cost
• KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
• CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios
• MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Models
• General-Reasoner: Advancing LLM Reasoning Across All Domains
• Acting Less is Reasoning More! Teaching Model to Act Efficiently
• BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
• SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction
• SMART: Self-Aware Agent for Tool Overuse Mitigation
• BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis
• LLM Inference Unveiled: Survey and Roofline Model Insights
• DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
• GAIA: a benchmark for General AI Assistants
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning
• Counterfactually Auditable Lifecycle Certification for Autonomous Agents
• ToolQA: A Dataset for LLM Question Answering with External Tools
• WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
• Training Verifiers to Solve Math Word Problems
• Measuring Mathematical Problem Solving With the MATH Dataset
• Scaling Laws for Neural Language Models
• StepTool: Enhancing Multi-Step Tool Usage in LLMs via Step-Grained Reinforcement Learning

Discussions

• Qisheng Su, Shiting Huang, Zhen Fang, Ziyan Chen, Zehui Chen, Feng Zhao: Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning https://arxiv.org/abs/2604.05404 https://arxiv.or [bsky, 0 points, 0 comments]

Related

• Dissecting Tool-Integrated Reasoning: An Empirical Study and Analysis
• OckBench: Measuring the Efficiency of LLM Reasoning
• Efficient Tool Use with Chain-of-Abstraction Reasoning
• An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint
• The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute