OCD: Learning to Overfit with Conditional Diffusion Models

2022/10/02 by Shahar Shlomo Lutati, Lutati, Shahar, Lior Wolf +1 · 2 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.2210.00471

openalex publication_date 2022/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a dynamic model in which the weights are conditioned on an input sample x and are learned to match those that would be obtained by finetuning a base model on x and its label y. This mapping between an input sample and network weights is approximated by a denoising diffusion model. The diffusion model we employ focuses on modifying a single layer of the base model and is conditioned on the input, activations, and output of this layer. Since the diffusion model is stochastic in nature, multiple initializations generate different networks, forming an ensemble, which leads to further improvements. Our experiments demonstrate the wide applicability of the method for image classification, 3D reconstruction, tabular data, speech separation, and natural language processing. Our code is available at https://github.com/ShaharLutatiPersonal/OCD

Cited by

• Learning an Efficient Optimizer via Hybrid-Policy Sub-Trajectory Balance
• GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes

Related

• Perception Prioritized Training of Diffusion Models
• APT: Adaptive Personalized Training for Diffusion Models with Limited Data
• Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models
• How Does Diffusion Influence Pretrained Language Models on Out-of-Distribution Data?
• FIND: Fine-tuning Initial Noise Distribution with Policy Optimization for Diffusion Models