Deep reinforcement learning for weakly coupled MDP's with continuous actions

2024/06/03 by Robledo, Francisco, Ayesta, Urtzi, Avrachenkov, Konstantin
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)

paper · doi:10.48550/arxiv.2406.01099

Abstract

This paper introduces the Lagrange Policy for Continuous Actions (LPCA), a reinforcement learning algorithm specifically designed for weakly coupled MDP problems with continuous action spaces. LPCA addresses the challenge of resource constraints dependent on continuous actions by introducing a Lagrange relaxation of the weakly coupled MDP problem within a neural network framework for Q-value computation. This approach effectively decouples the MDP, enabling efficient policy learning in resource-constrained environments. We present two variations of LPCA: LPCA-DE, which utilizes differential evolution for global optimization, and LPCA-Greedy, a method that incrementally and greadily selects actions based on Q-value gradients. Comparative analysis against other state-of-the-art techniques across various settings highlight LPCA's robustness and efficiency in managing resource allocation while maximizing rewards.

Related

• Weakly Coupled Deep Q-Networks
• Compatible Value Gradients for Reinforcement Learning of Continuous Deep\n Policies
• Deep Reinforcement Learning with Adjustments
• Randomized Policy Learning for Continuous State and Action MDPs
• Continuous control with deep reinforcement learning