Adaptive LLM Routing under Budget Constraints

2025/08/28 by Pranoy Panda, Panda, Pranoy, Raghav Magazine +7 · 17 voices · 3 citations
Computer Science · #Mobile Crowdsensing and Crowdsourcing #Multimodal Machine Learning Applications #Topic Modeling #cs.LG

paper · pdf · doi:10.48550/arxiv.2508.21141

openalex publication_date 2025/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have revolutionized natural language processing, but their varying capabilities and costs pose challenges in practical applications. LLM routing addresses this by dynamically selecting the most suitable LLM for each query/task. Previous approaches treat this as a supervised learning problem, assuming complete knowledge of optimal query-LLM pairings. However, real-world scenarios lack such comprehensive mappings and face evolving user queries. We thus propose to study LLM routing as a contextual bandit problem, enabling adaptive decision-making using bandit feedback without requiring exhaustive inference across all LLMs for all queries (in contrast to supervised routing). To address this problem, we develop a shared embedding space for queries and LLMs, where query and LLM embeddings are aligned to reflect their affinity. This space is initially learned from offline human preference data and refined through online bandit feedback. We instantiate this idea through Preference-prior Informed Linucb fOr adaptive rouTing (PILOT), a novel extension of LinUCB. To handle diverse user budgets for model routing, we introduce an online cost policy modeled as a multi-choice knapsack problem, ensuring resource-efficient routing.

Citations

• Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
• LoRA: Low-Rank Adaptation of Large Language Models
• Distilling the Knowledge in a Neural Network

Cited by

• WISERouter: LLM Routing with Workload Budget Constraint
• Reference-Free Rating of LLM Responses via Latent Information
• One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection

Discussions

• Adaptive LLM routing under budget constraints [hn, 206 points, 78 comments]
• 93% производительности GPT-4 при 1/4 стоимости: маршрутизация LLM со слабым обратной связью бандита #ai #gpt #llm [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback https://arxiv.org/abs/2508.21141 (https://news.ycombinator.com/item?id=45094421) [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback https://arxiv.org/abs/2508.21141 (https://news.ycombinator.com/item?id=45094421) [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback #HackerNews https://arxiv.org/abs/2508.21141 [bsky, 0 points, 0 comments]
• Adaptive LLM routing under budget constraints https://arxiv.org/abs/2508.21141 https://news.ycombinator.com/item?id=45094421 [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback https://arxiv.org/abs/2508.21141 [bsky, 0 points, 0 comments]
• Adaptive LLM routing under budget constraints View Article | Join the HN Conversation Summary of HN discussion 🧵👇 #hacker-news [bsky, 0 points, 1 comments]
• Adaptive LLM routing under budget constraints https://arxiv.org/abs/2508.21141 [comments] [188 points] [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback #ai #gpt #llm [bsky, 0 points, 0 comments]
• ⚡ Hackernews Top story: 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback [bsky, 0 points, 0 comments]
• Adaptive LLM routing under budget constraints https://arxiv.org/abs/2508.21141 (http://news.ycombinator.com/item?id=45094421) [bsky, 0 points, 0 comments]
• 最近、人工知能の研究で面白い論文が出ています。 この論文では、高性能な大規模言語モデル「GPT-4」と同じような性能を、コストのわずか1/4で実現する方法が紹介されています。 これは、複数の小さなモデルを組み合わせて使う「ルーティング」という手法と、学習効率を高める新しい方法を用いることで可能になったようです。 この研究は、将来的に大規模言語モデルの利用をより身近なものにする可能性がありますね。 [bsky, 0 points, 0 comments]
• https://arxiv.org/abs/2508.21141 予算制約下での適応的LLMルーティングに関する論文です。 大規模言語モデル（LLM）の効率的な利用を目指しています。 詳細な内容はリンク先で確認できます。 [bsky, 0 points, 0 comments]
• 93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback view on hacker news [bsky, 0 points, 0 comments]
• 📰 Adaptive LLM routing under budget constraints 🔗 https://arxiv.org/abs/2508.21141 💬 Discuss on HN [bsky, 0 points, 0 comments]

Related

• LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing
• WISERouter: LLM Routing with Workload Budget Constraint
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
• Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolution
• MixLLM: Dynamic Routing in Mixed Large Language Models