2025/03/01 by Junjie Sheng, Jiehao Wu, Sheng, Junjie +14
Computer Science · Engineering · #Cloud Computing and Resource Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Scheduling and Optimization Algorithms
paper · pdf · doi:10.48550/arxiv.2503.00537
openalex publication_date 2025/03/01 · openalex created_date 2025/10/12 · openalex updated_date 2026/07/28
Recent advancements in reinforcement learning (RL) have shown promise for optimizing virtual machine scheduling (VMS) in small-scale clusters. The utilization of RL to large-scale cloud computing scenarios remains notably constrained. This paper introduces a scalable RL framework, called Cluster Value Decomposition Reinforcement Learning (CVD-RL), to surmount the scalability hurdles inherent in large-scale VMS. The CVD-RL framework innovatively combines a decomposition operator with a look-ahead operator to adeptly manage representation complexities, while complemented by a Top-k filter operator that refines exploration efficiency. Different from existing approaches limited to clusters of 10 or fewer physical machines (PMs), CVD-RL extends its applicability to environments encompassing up to 50 PMs. Furthermore, the CVD-RL framework demonstrates generalization capabilities that surpass contemporary SOTA methodologies across a variety of scenarios in empirical studies. This breakthrough not only showcases the framework's exceptional scalability and performance but also represents a significant leap in the application of RL for VMS within complex, large-scale cloud infrastructures. The code is available at https://anonymous.4open.science/r/marl4sche-D0FE.