2021/01/01 by Youri Coppens, Denis Steckelmacher, Catholijn M. Jonker +1
Computer Science · #Adversarial Robustness in Machine Learning #Artificial intelligence #Benchmark (surveying) #Computer science #Explainable Artificial Intelligence (XAI) #Interpretability #Learning classifier system #Leverage (statistics) #Machine learning #Process (computing) #Programming language #Reinforcement Learning in Robotics #Reinforcement learning #Set (abstract data type) #cs.AI #cs.LG
paper · pdf · doi:10.1007/978-3-030-73959-1_15
published as Trustworthy AI - Integrating Learning, Optimization and Reasoning (2021), Lecture Notes in Computer Science, vol. 12641, pp. 163-179 · 17 pages, 4 figures. The final authenticated publication is available online at https://doi.org/10.1007/978-3-030-73959-1_15
openalex publication_date 2021/01/01 · arxiv created 2021/06/10 · arxiv updated 2021/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Today's advanced Reinforcement Learning algorithms produce black-box policies, that are often difficult to interpret and trust for a person. We introduce a policy distilling algorithm, building on the CN2 rule mining algorithm, that distills the policy into a rule-based decision system. At the core of our approach is the fact that an RL process does not just learn a policy, a mapping from states to actions, but also produces extra meta-information, such as action values indicating the quality of alternative actions. This meta-information can indicate whether more than one action is near-optimal for a certain state. We extend CN2 to make it able to leverage knowledge about equally-good actions to distill the policy into fewer rules, increasing its interpretability by a person. Then, to ensure that the rules explain a valid, non-degenerate policy, we introduce a refinement algorithm that fine-tunes the rules to obtain good performance when executed in the environment. We demonstrate the applicability of our algorithm on the Mario AI benchmark, a complex task that requires modern reinforcement learning algorithms including neural networks. The explanations we produce capture the learned policy in only a few rules, that allow a person to understand what the black-box agent learned. Source code: https://gitlab.ai.vub.ac.be/yocoppen/svcn2