Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images

2023/05/12 by Muhammad Usman Akbar, Akbar, Muhammad Usman, Wuhao Wang +3 · 4 citations
Computer Science · Medicine · #Advanced Neuroimaging Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Radiomics and Machine Learning in Medical Imaging #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2305.07644

openalex publication_date 2023/05/12 · openalex created_date 2023/05/18 · openalex updated_date 2026/07/28

Abstract

Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by GANs, diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as FID and IS are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray pneumonia dataset, to synthesize brain MRI and chest x-ray images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images.

Cited by

• SkinGenBench: Generative Model and Preprocessing Effects for Synthetic Dermoscopic Augmentation in Melanoma Diagnosis
• Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Models
• Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging?
• Benchmarking GANs, Diffusion Models, and Flow Matching for T1w-to-T2w MRI Translation

Related

• Spot the fake lungs: Generating Synthetic Medical Images using Neural Diffusion Models
• Diffusion-Based Approaches in Medical Image Generation and Analysis
• Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation
• DiNO-Diffusion. Scaling Medical Diffusion via Self-Supervised Pre-Training
• Unconditional Latent Diffusion Models Memorize Patient Imaging Data: Implications for Openly Sharing Synthetic Data