Neural Network Diffusion

2024/02/20 by Kai Wang, Wang, Kai, Dongwen Tang +17 · 1 voice · 15 citations
Computer Science · #Artificial intelligence #Artificial neural network #Computer science #Diffusion #Neural Networks and Applications #Physics #Thermodynamics #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2402.13144

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also generate high-performing neural network parameters. Our approach is simple, utilizing an autoencoder and a diffusion model. The autoencoder extracts latent representations of a subset of the trained neural network parameters. Next, a diffusion model is trained to synthesize these latent representations from random noise. This model then generates new representations, which are passed through the autoencoder's decoder to produce new subsets of high-performing network parameters. Across various architectures and datasets, our approach consistently generates models with comparable or improved performance over trained networks, with minimal additional cost. Notably, we empirically find that the generated models are not memorizing the trained ones. Our results encourage more exploration into the versatile use of diffusion models. Our code is available \hrefhttps://github.com/NUS-HPC-AI-Lab/Neural-Network-Diffusionhere.

Cited by

• PILD: Physics-Informed Learning via Diffusion
• Weight Space Representation Learning via Neural Field Adaptation
• On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
• Delta Activations: A Representation for Finetuned Large Language Models
• Learning with springs and sticks
• Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios
• Reimagining Parameter Space Exploration with Diffusion Models
• Generative Modeling of Weights: Generalization or Memorization?
• Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
• Symmetry in Neural Network Parameter Spaces
• DiffPINN: Generative diffusion-initialized physics-informed neural networks for accelerating seismic wavefield representation
• Fast Kernel-Space Diffusion for Remote Sensing Pansharpening
• Generative Adaptation of Dynamics to Environmental Shifts via Weight-space Diffusion
• NeuroGen: Neural Network Parameter Generation via Large Language Models

Discussions

• Neural Network Diffusion [hn, 223 points, 86 comments]

Related

• Neural Diffusion Models
• Understanding Diffusion Models: A Unified Perspective
• Image Neural Field Diffusion Models
• Stable Diffusion with Continuous-time Neural Network
• Diffusion Models as Masked Autoencoders