Temporal Concept Dynamics in Diffusion Models via Prompt-Conditioned Interventions

2025/12/09 by Gorgun, Ada, Sammani, Fawaz, Deligiannis, Nikos +2
Biochemistry, Genetics and Molecular Biology · Computer Science · #Cell Image Analysis Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis

paper · doi:10.48550/arxiv.2512.08486

openalex publication_date 2025/12/09 · openalex created_date 2025/12/11 · openalex updated_date 2026/07/28

Abstract

Diffusion models are usually evaluated by their final outputs, gradually denoising random noise into meaningful images. Yet, generation unfolds along a trajectory, and analyzing this dynamic process is crucial for understanding how controllable, reliable, and predictable these models are in terms of their success/failure modes. In this work, we ask the question: when does noise turn into a specific concept (e.g., age) and lock in the denoising trajectory? We propose PCI (Prompt-Conditioned Intervention) to study this question. PCI is a training-free and model-agnostic framework for analyzing concept dynamics through diffusion time. The central idea is the analysis of Concept Insertion Success (CIS), defined as the probability that a concept inserted at a given timestep is preserved and reflected in the final image, offering a way to characterize the temporal dynamics of concept formation. Applied to several state-of-the-art text-to-image diffusion models and a broad taxonomy of concepts, PCI reveals diverse temporal behaviors across diffusion models, in which certain phases of the trajectory are more favorable to specific concepts even within the same concept type. These findings also provide actionable insights for text-driven image editing, highlighting when interventions are most effective without requiring access to model internals or training, and yielding quantitatively stronger edits that achieve a balance of semantic accuracy and content preservation than strong baselines. Code is available at: https://adagorgun.github.io/PCI-Project/

Citations

• Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models
• Emergence and Evolution of Interpretable Concepts in Diffusion Models
• SmolVLM: Redefining small and efficient multimodal models
• Dissecting and Mitigating Diffusion Bias via Mechanistic Interpretability
• Interpretable Generative Models through Post-hoc Concept Bottlenecks
• TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation
• Qwen2.5-VL Technical Report
• ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
• SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders
• Vision-Language Models Do Not Understand Negation
• Stable Flow: Vital Layers for Training-Free Image Editing
• Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models
• LLaVA-OneVision: Easy Visual Task Transfer
• TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing
• Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• The Hidden Language of Diffusion Models
• Diffusion Self-Guidance for Controllable Image Generation
• HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models
• Structure and Content-Guided Video Synthesis with Diffusion Models
• Scalable Diffusion Models with Transformers
• Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models
• What the DAAM: Interpreting Stable Diffusion Using Cross Attention
• Flow Matching for Generative Modeling
• Building Normalizing Flows with Stochastic Interpolants
• Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
• Prompt-to-Prompt Image Editing with Cross Attention Control
• Classifier-Free Diffusion Guidance
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• High-Resolution Image Synthesis with Latent Diffusion Models
• Diffusion Models Beat GANs on Image Synthesis
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generations
• T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation

Related

• How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
• Emergence and Evolution of Interpretable Concepts in Diffusion Models
• When Are Concepts Erased From Diffusion Models?
• Unveiling Concept Attribution in Diffusion Models
• Concept Reachability in Diffusion Models: Beyond Dataset Constraints