SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

2020/12/24 by Xiangjun Wang, Wang, Xiangjun, Junxiao Song +21 · 3 citations
Computer Science · Social Sciences · #Artificial Intelligence in Games #Digital Games and Media #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2012.13169

openalex publication_date 2020/12/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a remarkable milestone demonstrating what deep reinforcement learning can achieve in complex Real-Time Strategy (RTS) games. However, the complexities of the game, algorithms and systems, and especially the tremendous amount of computation needed are big obstacles for the community to conduct further research in this direction. We propose a deep reinforcement learning agent, StarCraft Commander (SCC). With order of magnitude less computation, it demonstrates top human performance defeating GrandMaster players in test matches and top professional players in a live event. Moreover, it shows strong robustness to various human strategies and discovers novel strategies unseen from human plays. In this paper, we will share the key insights and optimizations on efficient imitation learning and reinforcement learning for StarCraft II full game.

Citations

• High-Dimensional Continuous Control Using Generalized Advantage\n Estimation
• Playing Atari with Deep Reinforcement Learning
• Episodic Exploration for Deep Deterministic Policies: An Application to\n StarCraft Micromanagement Tasks
• StarCraft II: A New Challenge for Reinforcement Learning
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Relational Deep Reinforcement Learning
• TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game
• Dota 2 with Large Scale Deep Reinforcement Learning
• TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game

Cited by

• In-Context Compositional Q-Learning for Offline Reinforcement Learning
• TacticCraft: Natural Language-Driven Tactical Adaptation for StarCraft II
• Reinforcement Learning via Conservative Agent for Environments with Random Delays

Related

• StarCraft II: A New Challenge for Reinforcement Learning
• On Reinforcement Learning for Full-length Game of StarCraft
• Asynchronous Advantage Actor-Critic Agent for Starcraft II
• Learning Macromanagement in StarCraft from Replays using Deep Learning
• Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning