Nepotistically Trained Generative-AI Models Collapse

2023/11/20 by Matyas Bohacek, Hany Farid · 3 voices · 2 citations
Computer Science · #cs.AI #cs.CV

paper · pdf

arxiv published 2023/11/20 · arxiv updated 2025/03/29

Abstract

Trained on massive amounts of human-generated content, AI-generated image synthesis is capable of reproducing semantically coherent images that match the visual appearance of its training data. We show that when retrained on even small amounts of their own creation, these generative-AI models produce highly distorted images. We also show that this distortion extends beyond the text prompts used in retraining, and that once affected, the models struggle to fully heal even after retraining on only real images.

Cited by

• A Markovian View of Iterative-Feedback Loops in Image Generative Models: Neural Resonance and Model Collapse
• From Collapse to Improvement: Statistical Perspectives on the Evolutionary Dynamics of Iterative Training on Contaminated Sources

Discussions

• “Nepotistically Trained Generative-AI Models Collapse” asserts that AI (artificial intelligence) image synthesis programs, when retrained on even small amounts of their own creation, produce highly di [bsky, 2 points, 0 comments]
• This was the first paper I read about it. References might contain earlier studies arxiv.org/abs/2311.12202 [bsky, 1 points, 3 comments]
• "For their own self preservation, AI models need to be able to distinguish between real and fake content." -Hany Farid arxiv.org/abs/2311.12202 [bsky, 0 points, 0 comments]

Related

• On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling
• The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better
• The Generative AI Paradox: "What It Can Create, It May Not Understand"
• Regeneration Based Training-free Attribution of Fake Images Generated by Text-to-Image Generative Models
• AI-Generated Images as Data Source: The Dawn of Synthetic Era