aMUSEd: An Open MUSE Reproduction

2024/01/03 by Suraj Patil, Patil, Suraj, William Berman +5 · 7 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Video Analysis and Summarization

paper · pdf · doi:10.48550/arxiv.2401.01808

openalex publication_date 2024/01/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present aMUSEd, an open-source, lightweight masked image model (MIM) for text-to-image generation based on MUSE. With 10 percent of MUSE's parameters, aMUSEd is focused on fast image generation. We believe MIM is under-explored compared to latent diffusion, the prevailing approach for text-to-image generation. Compared to latent diffusion, MIM requires fewer inference steps and is more interpretable. Additionally, MIM can be fine-tuned to learn additional styles with only a single image. We hope to encourage further exploration of MIM by demonstrating its effectiveness on large-scale text-to-image generation and releasing reproducible training code. We also release checkpoints for two models which directly produce images at 256x256 and 512x512 resolutions.

Cited by

• dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Models
• EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing
• Deconstructing Generative Diversity: An Information Bottleneck Analysis of Discrete Latent Generative Models
• Accelerating Inference of Masked Image Generators via Reinforcement Learning
• Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding
• Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings
• NEP: Autoregressive Image Editing via Next Editing Token Prediction

Related

• Muse: Text-To-Image Generation via Masked Generative Transformers
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model
• Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis
• Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
• MarkovGen: Structured Prediction for Efficient Text-to-Image Generation