Q-Networks for Binary Vector Actions

2015/12/04 by Naoto Yoshida, Yoshida, Naoto
Computer Science · #Adaptive Dynamic Programming Control #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Reinforcement Learning in Robotics #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.1512.01332

9 pages, 5 figures, accepted for Deep Reinforcement Learning Workshop, NIPS 2015

arxiv created 2015/12/04 · openalex publication_date 2015/12/04 · arxiv updated 2015/12/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper reinforcement learning with binary vector actions was investigated. We suggest an effective architecture of the neural networks for approximating an action-value function with binary vector actions. The proposed architecture approximates the action-value function by a linear function with respect to the action vector, but is still non-linear with respect to the state input. We show that this approximation method enables the efficient calculation of greedy action selection and softmax action selection. Using this architecture, we suggest an online algorithm based on Q-learning. The empirical results in the grid world and the blocker task suggest that our approximation architecture would be effective for the RL problems with large discrete action sets.

Citations

• Playing Atari with Deep Reinforcement Learning
• Continuous control with deep reinforcement learning

Related

• Discrete linear-complexity reinforcement learning in continuous action\n spaces for Q-learning algorithms
• Stochastic Q-learning for Large Discrete Action Spaces
• Deep Reinforcement Learning with Adjustments
• Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces
• Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement Learning