Composing Task Knowledge with Modular Successor Feature Approximators

2023/01/28 by Wilka Carvalho, Angelos Filos, Carvalho, Wilka +7 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2301.12305

openalex publication_date 2023/01/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge and behavior. SF&GPI works by having an agent learn predictive representations (SFs) that can be combined for transfer to new tasks with GPI. However, to be effective this approach requires state features that are useful to predict, and these state-features are typically hand-designed. In this work, we present a novel neural network architecture, "Modular Successor Feature Approximators" (MSFA), where modules both discover what is useful to predict, and learn their own predictive representations. We show that MSFA is able to better generalize compared to baseline architectures for learning SFs and modular architectures

Cited by

• A Unified Theory of Compositionality, Modularity, and Interpretability in Markov Decision Processes

Related

• Combining Behaviors with the Successor Features Keyboard
• Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learning
• Successor Feature Sets: Generalizing Successor Representations Across\n Policies
• Successor Feature Neural Episodic Control
• Universal Successor Features Approximators