Re-understanding Finite-State Representations of Recurrent Policy Networks

2020/06/06 by Danesh, Mohamad H., Koul, Anurag, Fern, Alan +1 · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2006.03745

Abstract

We introduce an approach for understanding control policies represented as recurrent neural networks. Recent work has approached this problem by transforming such recurrent policy networks into finite-state machines (FSM) and then analyzing the equivalent minimized FSM. While this led to interesting insights, the minimization process can obscure a deeper understanding of a machine's operation by merging states that are semantically distinct. To address this issue, we introduce an analysis approach that starts with an unminimized FSM and applies more-interpretable reductions that preserve the key decision points of the policy. We also contribute an attention tool to attain a deeper understanding of the role of observations in the decisions. Our case studies on 7 Atari games and 3 control benchmarks demonstrate that the approach can reveal insights that have not been previously noticed.

Cited by

• Know your Trajectory -- Trustworthy Reinforcement Learning deployment through Importance-Based Trajectory Analysis

Related

• Learning Finite State Representations of Recurrent Policy Networks
• Neural Co-state Policies: Structuring Hidden States in Recurrent Reinforcement Learning
• A2: Extracting Cyclic Switchings from DOB-nets for Rejecting Excessive Disturbances
• Verifiable RNN-Based Policies for POMDPs Under Temporal Logic Constraints
• Recurrent Predictive State Policy Networks