2020/08/28 by Sanxing Chen, Xiaodong Liu, Chen, Sanxing +9 · 3 citations
Computer Science · Decision Sciences · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2008.12813
openalex publication_date 2020/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood. Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block. We further design a masked entity prediction task to balance information from the relational context and the source entity itself. Experimental results show that HittER achieves new state-of-the-art results on multiple link prediction datasets. We additionally propose a simple approach to integrate HittER into BERT and demonstrate its effectiveness on two Freebase factoid question answering datasets.