GeckOpt: LLM System Efficiency via Intent-Based Tool Selection

2024/04/24 by Michael Fore, Fore, Michael, Simranjit Singh +3 · 4 citations
Computer Science · Engineering · #Advanced Surface Polishing Techniques #Artificial Intelligence (cs.AI) #Digital Rights Management and Security #FOS: Computer and information sciences #Machine Learning (cs.LG) #Manufacturing Process and Optimization

paper · pdf · doi:10.48550/arxiv.2404.15804

openalex publication_date 2024/04/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this preliminary study, we investigate a GPT-driven intent-based reasoning approach to streamline tool selection for large language models (LLMs) aimed at system efficiency. By identifying the intent behind user prompts at runtime, we narrow down the API toolset required for task execution, reducing token consumption by up to 24.6%. Early results on a real-world, massively parallel Copilot platform with over 100 GPT-4-Turbo nodes show cost reductions and potential towards improving LLM-based system efficiency.

Cited by

• GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chaining
• Automated Multi-Agent Workflows for RTL Design
• LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions
• MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models

Related

• iServe: An Intent-based Serving System for LLMs
• ToolTalk: Evaluating Tool-Usage in a Conversational Setting
• An LLM-Tool Compiler for Fused Parallel Function Calling
• Large Language Models as Tool Makers
• Counterfactually Auditable Lifecycle Certification for Autonomous Agents