Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing
2025/08/18 by Yiqun Zhang, Zhang, Yiqun, Hao Li +11 · 11 voices · 9 citations
Computer Science · Medicine · #Topic Modeling #Natural Language Processing Techniques #Artificial Intelligence in Healthcare and Education
paper · pdf · doi:10.1145/3772429.3772445
Abstract
Balancing performance and efficiency is a central challenge in large language model (LLM) advancement. GPT-5 addresses this with test-time routing, dynamically assigning queries to either an efficient or a high-capacity model during inference. In this work, we present Avengers-Pro, a test-time routing framework that ensembles LLMs of varying capacities and efficiencies, providing a unified solution for all performance-efficiency tradeoffs. The Avengers-Pro embeds and clusters incoming queries, then routes each to the most suitable model based on a performance-efficiency score. Across 6 challenging benchmarks and 8 leading models—including GPT-5-medium, Gemini-2.5-pro, and Claude-opus-4.1—Avengers-Pro achieves state-of-the-art results: by varying a performance-efficiency trade-off parameter, it can surpass the strongest single model (GPT-5-medium) by +7% in average accuracy. Moreover, it can match the average accuracy of the strongest single model at 27% lower cost, and reach ∼ 90% of that performance at 63% lower cost. Last but not least, it achieves a Pareto frontier, consistently yielding the highest accuracy for any given cost, and the lowest cost for any given accuracy, among all single models. Code is available at https://github.com/ZhangYiqun018/AvengersPro.
Citations
Cited by
Discussions
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing [hn, 130 points, 28 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing https://arxiv.org/abs/2508.12631 (https://news.ycombinator.com/item?id=44985278) [bsky, 0 points, 0 comments]
- https://bsky.app/profile/hackernews.com.web.brid.gy/post/3lwyxfb7ww4y2 [bsky, 0 points, 0 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing [bsky, 0 points, 0 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing #HackerNews https://arxiv.org/abs/2508.12631 [bsky, 0 points, 0 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing https://arxiv.org/abs/2508.12631 (https://news.ycombinator.com/item?id=44985278) [bsky, 0 points, 0 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing https://arxiv.org/abs/2508.12631 [bsky, 0 points, 0 comments]
- ⚡ Hackernews Top story: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing [bsky, 0 points, 0 comments]
- Original paper on how a router balances performance and efficiency: arxiv.org/abs/2508.12631 #llms #ai #llmrouting #oss [bsky, 0 points, 0 comments]
- https://arxiv.org/abs/2508.12631 GPT-5を超える、より安価で優れたLLMの実現に関する論文です。 パフォーマンスと効率が最適化されたルーティングを通じてコスト削減を目指しています。 今後のLLM開発の方向性を示唆する内容となっています。 [bsky, 0 points, 0 comments]
- Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing https://arxiv.org/abs/2508.12631 https://news.ycombinator.com/item?id=44985278 [bsky, 0 points, 0 comments]
Related