2025/03/10 by Chengwei Zhang, Zhou, Kailing, Furui Zhan +6
Social Sciences · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Policy Transfer and Learning
paper · doi:10.48550/arxiv.2503.07678
openalex publication_date 2025/03/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent advances in Multi-Agent Reinforcement Learning (MARL) have demonstrated significant potential for adaptive traffic signal control. However, existing MARL approaches face dual challenges: Complete parameter sharing among agents leads to insufficient diversity in policy networks, while treating agents as heterogeneous entities (with non-shared parameters) better adapts to intersection heterogeneity but introduces training inefficiency and parameter explosion in large-scale road networks. Balancing agent performance with reduced computational resource consumption remains a critical challenge that demands urgent resolution. To address these issues, we propose the Hyper-Action Multi-Head Proximal Policy Optimization (HAMH-PPO) method, which enhances personalized representation capabilities through value functions. This approach constructs K shared value function libraries to provide differentiated value estimation for traffic networks, while employing a hyper-network to dynamically generate adaptive weights for these libraries across different intersections. Consequently, a single shared network can learn diverse control strategies that accommodate varying observations. The experimental results showed that HAMH-PPO outperforms traditional traffic signal methods by 39.44% and achieves a 29.8% improvement over the suboptimal solution in complex, large-scale synthetic road networks. Crucially, HAMH-PPO maintains the benefits of parameter sharing while optimizing algorithmic performance and reducing computational costs.