Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning

2023/02/04 by Xiangrong Zhu, Guangyao Li, Zhu, Xiangrong +3 · 6 citations
Computer Science · Psychology · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mental Health via Writing #Privacy-Preserving Technologies in Data #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.2302.02069

openalex publication_date 2023/02/04 · openalex created_date 2023/02/09 · openalex updated_date 2026/07/28

Abstract

Federated Learning (FL) recently emerges as a paradigm to train a global machine learning model across distributed clients without sharing raw data. Knowledge Graph (KG) embedding represents KGs in a continuous vector space, serving as the backbone of many knowledge-driven applications. As a promising combination, federated KG embedding can fully take advantage of knowledge learned from different clients while preserving the privacy of local data. However, realistic problems such as data heterogeneity and knowledge forgetting still remain to be concerned. In this paper, we propose FedLU, a novel FL framework for heterogeneous KG embedding learning and unlearning. To cope with the drift between local optimization and global convergence caused by data heterogeneity, we propose mutual knowledge distillation to transfer local knowledge to global, and absorb global knowledge back. Moreover, we present an unlearning method based on cognitive neuroscience, which combines retroactive interference and passive decay to erase specific knowledge from local clients and propagate to the global model by reusing knowledge distillation. We construct new datasets for assessing realistic performance of the state-of-the-arts. Extensive experiments show that FedLU achieves superior results in both link prediction and knowledge forgetting.

Cited by

• Rethinking Knowledge Distillation in Collaborative Machine Learning: Memory, Knowledge, and Their Interactions
• REMISVFU: Vertical Federated Unlearning via Representation Misdirection for Intermediate Output Feature
• Federated Unlearning in the Wild: Rethinking Fairness and Data Discrepancy
• Efficient Knowledge Graph Unlearning with Zeroth-order Information
• A Comprehensive Data-centric Overview of Federated Graph Learning
• Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion

Related

• Federated Knowledge Graph Unlearning via Diffusion Model
• Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Data
• Handling Data Heterogeneity in Federated Learning via Knowledge Distillation and Fusion
• Preservation of the Global Knowledge by Not-True Distillation in Federated Learning
• FedGKD: Toward Heterogeneous Federated Learning via Global Knowledge Distillation