2021/09/28 by Kin-Ho Lam, Lam, Kin-Ho, Zhengxian Lin +14
Computer Science · Economics, Econometrics and Finance · #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Sports Analytics and Performance #cs.AI
paper · pdf · doi:10.48550/arxiv.2109.13978
arxiv created 2021/09/28 · openalex publication_date 2021/09/28 · arxiv updated 2021/09/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Enabling humans to identify potential flaws in an agent's decision making is an important Explainable AI application. We consider identifying such flaws in a planning-based deep reinforcement learning (RL) agent for a complex real-time strategy game. In particular, the agent makes decisions via tree search using a learned model and evaluation function over interpretable states and actions. This gives the potential for humans to identify flaws at the level of reasoning steps in the tree, even if the entire reasoning process is too complex to understand. However, it is unclear whether humans will be able to identify such flaws due to the size and complexity of trees. We describe a user interface and case study, where a small group of AI experts and developers attempt to identify reasoning flaws due to inaccurate agent learning. Overall, the interface allowed the group to identify a number of significant flaws of varying types, demonstrating the promise of this approach.