Compositionality for Recursive Neural Networks

2019/01/30 by Martha Lewis, Lewis, Martha · 1 citation
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Category Theory (math.CT) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Mathematics #Natural Language Processing Techniques #Neural and Evolutionary Computing (cs.NE) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1901.10723

openalex publication_date 2019/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Modelling compositionality has been a longstanding area of research in the field of vector space semantics. The categorical approach to compositionality maps grammar onto vector spaces in a principled way, but comes under fire for requiring the formation of very high-dimensional matrices and tensors, and therefore being computationally infeasible. In this paper I show how a linear simplification of recursive neural tensor network models can be mapped directly onto the categorical approach, giving a way of computing the required matrices and tensors. This mapping suggests a number of lines of research for both categorical compositional vector space models of meaning and for recursive neural network models of compositionality.

Cited by

• Disentangling Hyperedges through the Lens of Category Theory

Related

• Compositionality decomposed: how do neural networks generalise?
• Evaluating Neural Word Representations in Tensor-Based Compositional\n Settings
• Modeling Compositionality with Multiplicative Recurrent Neural Networks
• A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition
• Developing a Foundation of Vector Symbolic Architectures Using Category Theory