L2E: Learning to Exploit Your Opponent

2021/02/18 by Zhe Wu, Wu, Zhe, Kai Li +12
Computer Science · Economics, Econometrics and Finance · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Computer Science and Game Theory (cs.GT) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Sports Analytics and Performance

paper · pdf · doi:10.48550/arxiv.2102.09381

openalex publication_date 2021/02/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Opponent modeling is essential to exploit sub-optimal opponents in strategic interactions. Most previous works focus on building explicit models to directly predict the opponents' styles or strategies, which require a large amount of data to train the model and lack adaptability to unknown opponents. In this work, we propose a novel Learning to Exploit (L2E) framework for implicit opponent modeling. L2E acquires the ability to exploit opponents by a few interactions with different opponents during training, thus can adapt to new opponents with unknown styles during testing quickly. We propose a novel opponent strategy generation algorithm that produces effective opponents for training automatically. We evaluate L2E on two poker games and one grid soccer game, which are the commonly used benchmarks for opponent modeling. Comprehensive experimental results indicate that L2E quickly adapts to diverse styles of unknown opponents.

Citations

• Learning to reinforcement learn
• Robust Adversarial Reinforcement Learning
• Continuous Adaptation via Meta-Learning in Nonstationary and Competitive\n Environments
• On First-Order Meta-Learning Algorithms
• Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
• Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning
• Wasserstein Robust Reinforcement Learning

Related

• Opponent Modeling in Deep Reinforcement Learning
• Learning to Model Opponent Learning
• Consistent Opponent Modeling in Imperfect-Information Games
• Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games
• Opponent Modeling in Multiplayer Imperfect-Information Games