Learning to Reason with Third-Order Tensor Products

2018/11/29 by Imanol Schlag, Jürgen Schmidhuber, Schlag, Imanol +1 · 4 citations
Computer Science · #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #Neural and Evolutionary Computing (cs.NE) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1811.12143

openalex publication_date 2018/11/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We combine Recurrent Neural Networks with Tensor Product Representations to learn combinatorial representations of sequential data. This improves symbolic interpretation and systematic generalisation. Our architecture is trained end-to-end through gradient descent on a variety of simple natural language reasoning tasks, significantly outperforming the latest state-of-the-art models in single-task and all-tasks settings. We also augment a subset of the data such that training and test data exhibit large systematic differences and show that our approach generalises better than the previous state-of-the-art.

Cited by

• Who invented deep residual learning?
• Memory and attention in deep learning
• A Simple Recurrent Unit with Reduced Tensor Product Representations
• HUBERT Untangles BERT to Improve Transfer across NLP Tasks
• A Perspective on Objects and Systematic Generalization in Model-Based RL

Related

• Can recursive neural tensor networks learn logical reasoning?
• RNNs Implicitly Implement Tensor Product Representations
• Continual Reasoning: Non-Monotonic Reasoning in Neurosymbolic AI using Continual Learning
• Tensor Logic: The Language of AI
• Discrete Dictionary-based Decomposition Layer for Structured Representation Learning