Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models

2023/02/08 by Shawn Shan, Jenna Cryan, Shan, Shawn +10 · 7 voices · 44 citations
Computer Science · Neuroscience · #Aesthetic Perception and Analysis #Aesthetics #Art #Artificial intelligence #Computer science #Generative Adversarial Networks and Image Synthesis #Generative grammar #Glaze #Gloss (optics) #Human–computer interaction #Mimicry #Music and Audio Processing #Robustness (evolution) #Style (visual arts) #Usability #Visual arts #cs.CR

paper · pdf · doi:10.48550/arxiv.2302.04222

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2023/02/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community. In particular, models can learn to mimic the artistic style of specific artists after "fine-tuning" on samples of their art. In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply "style cloaks" to their art before sharing online. These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist. In coordination with the professional artist community, we deploy user studies to more than 1000 artists, assessing their views of AI art, as well as the efficacy of our tool, its usability and tolerability of perturbations, and robustness across different scenarios and against adaptive countermeasures. Both surveyed artists and empirical CLIP-based scores show that even at low perturbation levels (p=0.05), Glaze is highly successful at disrupting mimicry under normal conditions (>92%) and against adaptive countermeasures (>85%).

Cited by

• Semantic Color Naturalness Breaker: Preventing Illegitimate Colorization via Content-Aware Color Priors
• How Professional Visual Artists are Negotiating Generative AI in the Workplace
• I2VShield: An Efficient Proactive Defense Framework against DiT-based Image-to-Video Models
• DeContext as Defense: Safe Image Editing in Diffusion Transformers
• Where is the Watermark? Interpretable Watermark Detection at the Block Level
• AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing
• HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models
• The Persistence of Cultural Memory: Investigating Multimodal Iconicity in Diffusion Models
• BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editing
• Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models
• AegisRF: Adversarial Perturbations Guided with Sensitivity for Protecting Intellectual Property of Neural Radiance Fields
• Prompt injections as a tool for preserving identity in GAI image descriptions
• SafeGuider: Robust and Practical Content Safety Control for Text-to-Image Models
• DIA: The Adversarial Exposure of Deterministic Inversion in Diffusion Models
• Vid-Freeze: Protecting Images from Malicious Image-to-Video Generation via Temporal Freezing
• VCE: Safe Autoregressive Image Generation via Visual Contrast Exploitation
• Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
• Immunizing Images from Text to Image Editing via Adversarial Cross-Attention
• SuMa: A Subspace Mapping Approach for Robust and Effective Concept Erasure in Text-to-Image Diffusion Models
• PromptFlare: Prompt-Generalized Defense via Cross-Attention Decoy in Diffusion-Based Inpainting
• Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
• Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model
• LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning
• An h-space Based Adversarial Attack for Protection Against Few-shot Personalization
• GIFT: Gradient-aware Immunization of diffusion models against malicious Fine-Tuning with safe concepts retention
• When and Where do Data Poisons Attack Textual Inversion?
• From Imitation to Innovation: The Emergence of AI Unique Artistic Styles and the Challenge of Copyright Protection
• Rethinking Data Protection in the (Generative) Artificial Intelligence Era
• Perpetuating Misogyny with Generative AI: How Model Personalization Normalizes Gendered Harm
• Is Perturbation-Based Image Protection Disruptive to Image Editing?
• Beyond Invisibility: Learning Robust Visible Watermarks for Stronger Copyright Protection
• Imago Obscura: An Image Privacy AI Co-pilot to Enable Identification and Mitigation of Risks
• Structure Disruption: Subverting Malicious Diffusion-Based Inpainting via Self-Attention Query Perturbation
• StyleGuard: Preventing Text-to-Image-Model-based Style Mimicry Attacks by Style Perturbations
• Signals of Provenance: Practices & Challenges of Navigating Indicators in AI-Generated Media for Sighted and Blind Individuals
• Visual Watermarking in the Era of Diffusion Models: Advances and Challenges
• "I Just Don't Want My Work Being Fed Into The AI Blender": Queer Artists on Refusing and Resisting Generative AI
• T2VAttack: Adversarial Attack on Text-to-Video Diffusion Models
• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle
• What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale
• Exploring Language Patterns of Prompts in Text-to-Image Generation and Their Impact on Visual Diversity
• ArtistAuditor: Auditing Artist Style Pirate in Text-to-Image Generation Models
• IPV-Bench: Benchmarking Image Protection Methods under Diverse Image-to-Video Generation Scenarios
• FPGA-Optimized Hardware Accelerator for Fast Fourier Transform and Singular Value Decomposition in AI
• Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking

Discussions

• GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models [lobsters, 16 points, 1 comments]
• glaze.cs.uchicago.edu arxiv.org/abs/2302.04222 Glaze is out (in beta afaik), but it's supposed to help artists prevent their art from being stolen for data scraping by AI assholes. A team at UChicago [bsky, 4 points, 2 comments]
• Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models [hn, 3 points, 2 comments]
• Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models [hn, 3 points, 0 comments]
• Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models [pdf] [hn, 1 points, 0 comments]
• This is the "extended technical report". Section 5 describes the design and approach, so it should in theory be independently reproducible. [bsky, 0 points, 0 comments]
• [1]: doi.org/10.48550/arX... [2]: doi.org/10.48550/arX... [3]: glaze.cs.uchicago.edu/update21.html [bsky, 0 points, 0 comments]

Related

• GLEAN: Generative Learning for Eliminating Adversarial Noise
• StyleProtect: Safeguarding Artistic Identity in Fine-tuned Diffusion Models
• Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI
• Organic or Diffused: Can We Distinguish Human Art from AI-generated Images?
• Measuring the Success of Diffusion Models at Imitating Human Artists