Compositional Vector Space Models for Knowledge Base Completion

2015/04/24 by Arvind Neelakantan, Neelakantan, Arvind, Benjamin Roth +3 · 14 citations
Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (stat.ML) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1504.06662

openalex publication_date 2015/04/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic feature, like bornIn(X,Z) -> containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recursive neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assemble a new dataset of over 52M relational triples, and show that our method improves over a traditional classifier by 11%, and a method leveraging pre-trained embeddings by 7%.

Citations

• Investigating the Role of Prior Disambiguation in Deep-learning Compositional Models of Meaning
• Neural Machine Translation by Jointly Learning to Align and Translate
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Cited by

• Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning
• Knowledge Graph Embedding with Iterative Guidance from Soft Rules
• Knowledge Graph Representation with Jointly Structural and Textual Encoding
• Improving Information Extraction by Acquiring External Evidence with\n Reinforcement Learning
• Compositional Learning of Relation Path Embedding for Knowledge Base Completion
• Link Prediction using Embedded Knowledge Graphs
• DuetGraph: Coarse-to-Fine Knowledge Graph Reasoning with Dual-Pathway Global-Local Fusion
• NLProlog: Reasoning with Weak Unification for Question Answering in\n Natural Language
• Traversing Knowledge Graphs in Vector Space
• Group Representation Theory for Knowledge Graph Embedding
• Path-based knowledge reasoning with textual semantic information for medical knowledge graph completion
• Neural Programmer: Inducing Latent Programs with Gradient Descent
• Reasoning over RDF Knowledge Bases using Deep Learning
• Two-dimensional Taxonomy for N-ary Knowledge Representation Learning Methods

Related

• Compositional Learning of Relation Path Embedding for Knowledge Base Completion
• Embedding Entities and Relations for Learning and Inference in Knowledge Bases
• Modeling Relation Paths for Representation Learning of Knowledge Bases
• Combining Rules and Embeddings via Neuro-Symbolic AI for Knowledge Base Completion
• Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations