2025/01/28 by Xiaolin Ni, Ni, Xinyi, Qiuyang Wang +4 · 1 citation
Business, Management and Accounting · Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Business Process Modeling and Analysis #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Scientific Computing and Data Management #Service-Oriented Architecture and Web Services #Software Engineering (cs.SE)
paper · pdf · doi:10.48550/arxiv.2501.16945
openalex publication_date 2025/01/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
LLM-based tool agents offer natural language interfaces, enabling users to seamlessly interact with computing services. While REST APIs are valuable resources for building such agents, they must first be transformed into AI-compatible tools. Automatically generating AI-compatible tools from REST API documents can greatly streamline tool agent development and minimize user learning curves. However, API documentation often suffers from a lack of standardization, inconsistent schemas, and incomplete information. To address these issues, we developed ToolFactory, an open-source pipeline for automating tool generation from unstructured API documents. To enhance the reliability of the developed tools, we implemented an evaluation method to diagnose errors. Furthermore, we built a knowledge base of verified tools, which we leveraged to infer missing information from poorly documented APIs. We developed the API Extraction Benchmark, comprising 167 API documents and 744 endpoints in various formats, and designed a JSON schema to annotate them. This annotated dataset was utilized to train and validate ToolFactory. The experimental results highlight the effectiveness of ToolFactory. We also demonstrated ToolFactory by creating a domain-specific AI agent for glycomaterials research. ToolFactory exhibits significant potential for facilitating the seamless integration of scientific REST APIs into AI workflows.