Training Diffusion Models with Federated Learning

2024/06/18 by de Goede, Matthijs, Cox, Bart, Decouchant, Jérémie · 2 citations
#Distributed #FOS: Computer and information sciences #I.2.11 #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)

paper · doi:10.48550/arxiv.2406.12575

Abstract

The training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies, raising concerns about privacy, copyright, and data authority due to their lack of transparency regarding training data. To ad-dress this issue, we propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data. Our approach adapts the Federated Averaging (FedAvg) algorithm to train a Denoising Diffusion Model (DDPM). Through a novel utilization of the underlying UNet backbone, we achieve a significant reduction of up to 74% in the number of parameters exchanged during training,compared to the naive FedAvg approach, whilst simultaneously maintaining image quality comparable to the centralized setting, as evaluated by the FID score.

Cited by

• FedSDA: Federated Stain Distribution Alignment for Non-IID Histopathological Image Classification
• Federated Flow Matching

Related

• Phoenix: A Federated Generative Diffusion Model
• FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Models
• FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models
• Federated Learning with Diffusion Models for Privacy-Sensitive Vision Tasks
• FedDifRC: Unlocking the Potential of Text-to-Image Diffusion Models in Heterogeneous Federated Learning