Playing FPS Games with Deep Reinforcement Learning

2016/09/18 by Guillaume Lample, Devendra Singh Chaplot, Lample, Guillaume +1 · 2 voices · 112 citations
Computer Science · #Architecture #Artificial Intelligence in Games #Artificial intelligence #Computer science #Computer security #Deep learning #Exploit #Feature (linguistics) #Human Pose and Action Recognition #Machine learning #Multimedia #Reinforcement Learning in Robotics #Reinforcement learning #Video game #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1609.05521

published in arXiv (Cornell University) 31(1), 2140-2146 (Cornell University) · The authors contributed equally to this work

openalex publication_date 2016/09/18 · arxiv created 2018/01/29 · arxiv updated 2018/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions. However, most of these games take place in 2D environments that are fully observable to the agent. In this paper, we present the first architecture to tackle 3D environments in first-person shooter games, that involve partially observable states. Typically, deep reinforcement learning methods only utilize visual input for training. We present a method to augment these models to exploit game feature information such as the presence of enemies or items, during the training phase. Our model is trained to simultaneously learn these features along with minimizing a Q-learning objective, which is shown to dramatically improve the training speed and performance of our agent. Our architecture is also modularized to allow different models to be independently trained for different phases of the game. We show that the proposed architecture substantially outperforms built-in AI agents of the game as well as humans in deathmatch scenarios.

Cited by

• Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems
• Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learning
• Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
• The StreetLearn Environment and Dataset
• Shaping Belief States with Generative Environment Models for RL
• Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping
• Compare and Select: Video Summarization with Multi-Agent Reinforcement Learning
• Learning Generalizable Visual Representations via Interactive Gameplay
• Discovering Differential Features: Adversarial Learning for Information Credibility Evaluation
• Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System
• Unity: A General Platform for Intelligent Agents
• Mastering Complex Control in MOBA Games with Deep Reinforcement Learning
• DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
• A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
• Off-Policy Actor-Critic with Shared Experience Replay
• Recurrent Off-policy Baselines for Memory-based Continuous Control
• UrzaGPT: LoRA-Tuned Large Language Models for Card Selection in Collectible Card Games
• Hierarchical Deep Deterministic Policy Gradient for Autonomous Maze Navigation of Mobile Robots
• ThermoRL:Structure-Aware Reinforcement Learning for Protein Mutation Design to Enhance Thermostability
• Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent
• Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning
• Zero-Shot Learning of Text Adventure Games with Sentence-Level Semantics
• Influence-aware Memory Architectures for Deep Reinforcement Learning
• Data-Driven Policy Mapping for Safe RL-based Energy Management Systems
• EgoMap: Projective mapping and structured egocentric memory for Deep RL
• An Introduction to Neural Architecture Search for Convolutional Networks
• Distributionally Robust Deep Q-Learning
• Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations
• Multi-agent Embodied AI: Advances and Future Directions
• Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships
• Do We Need Transformers to Play FPS Video Games?
• Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios
• Predicting Game Difficulty and Churn Without Players

Discussions

• Playing FPS Video Games with Deep Reinforcement Learning [hn, 61 points, 19 comments]
• Playing Doom with Deep Reinforcement Learning [hn, 3 points, 0 comments]

Related

• Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning
• Deep Reinforcement Learning for Doom using Unsupervised Auxiliary Tasks
• Reinforcement Learning Agent for a 2D Shooter Game
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Playing Doom with SLAM-Augmented Deep Reinforcement Learning