Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding

2023/04/23 by Yichi Zhang, Mingyang Chen, Zhang, Yichi +3 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Visual Attention and Saliency Detection

paper · pdf · doi:10.48550/arxiv.2304.11618

openalex publication_date 2023/04/23 · openalex created_date 2023/04/27 · openalex updated_date 2026/07/28

Abstract

Negative sampling (NS) is widely used in knowledge graph embedding (KGE), which aims to generate negative triples to make a positive-negative contrast during training. However, existing NS methods are unsuitable when multi-modal information is considered in KGE models. They are also inefficient due to their complex design. In this paper, we propose Modality-Aware Negative Sampling (MANS) for multi-modal knowledge graph embedding (MMKGE) to address the mentioned problems. MANS could align structural and visual embeddings for entities in KGs and learn meaningful embeddings to perform better in multi-modal KGE while keeping lightweight and efficient. Empirical results on two benchmarks demonstrate that MANS outperforms existing NS methods. Meanwhile, we make further explorations about MANS to confirm its effectiveness.

Cited by

• PGMEL: Policy Gradient-based Generative Adversarial Network for Multimodal Entity Linking
• Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion

Related

• Negative Sampling with Adaptive Denoising Mixup for Knowledge Graph Embedding
• MixKG: Mixing for harder negative samples in knowledge graph
• Diversified and Adaptive Negative Sampling on Knowledge Graphs
• Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion
• Entity Aware Negative Sampling with Auxiliary Loss of False Negative Prediction for Knowledge Graph Embedding