Crucible: Quantifying the Potential of Control Algorithms through LLM Agents

2025/10/21 by Lianchen Jia, Jia, Lianchen, Chaoyang Li +9
Computer Science · #AI-based Problem Solving and Planning #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2510.18491

openalex publication_date 2025/10/21 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

Control algorithms in production environments typically require domain experts to tune their parameters and logic for specific scenarios. However, existing research predominantly focuses on algorithmic performance under ideal or default configurations, overlooking the critical aspect of Tuning Potential. To bridge this gap, we introduce Crucible, an agent that employs an LLM-driven, multi-level expert simulation to turn algorithms and defines a formalized metric to quantitatively evaluate their Tuning Potential. We demonstrate Crucible's effectiveness across a wide spectrum of case studies, from classic control tasks to complex computer systems, and validate its findings in a real-world deployment. Our experimental results reveal that Crucible systematically quantifies the tunable space across different algorithms. Furthermore, Crucible provides a new dimension for algorithm analysis and design, which ultimately leads to performance improvements. Our code is available at https://github.com/thu-media/Crucible.

Citations

• Towards User-level QoE: Large-scale Practice in Personalized Optimization of Adaptive Video Streaming
• Beyond Interpretability: Exploring the Comprehensibility of Adaptive Video Streaming through Large Language Models
• A tutorial on open-source large language models for behavioral science
• BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
• Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations
• Generative Agents: Interactive Simulacra of Human Behavior
• A Survey of Large Language Models
• GPT-4 Technical Report
• Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• WebGPT: Browser-assisted question-answering with human feedback
• Hyper-Parameter Optimization: A Review of Algorithms and Applications
• Scaling Laws for Neural Language Models
• A Survey of Autonomous Driving: Common Practices and Emerging Technologies
• Learning Scheduling Algorithms for Data Processing Clusters
• OpenAI Gym
• A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles
• Practical Bayesian Optimization of Machine Learning Algorithms
• A review of techniques for parameter sensitivity analysis of environmental models
• Reinforcement Learning: An Introduction

Related

• ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertise
• Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents
• LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
• Credo: Declarative Control of LLM Pipelines via Beliefs and Policies
• Know the Ropes: A Heuristic Strategy for LLM-based Multi-Agent System Design