2016/12/09 by Han Zhang, Zhang, Han, Tao Xu +11 · 8 voices · 33 citations
Computer Science · #Computer Graphics and Visualization Techniques #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #cs.AI #cs.CV #stat.ML
paper · pdf · doi:10.48550/arxiv.1612.03242
openalex publication_date 2016/12/09 · openalex created_date 2022/09/25 · openalex updated_date 2026/07/28
Synthesizing high-quality images from text descriptions is a challenging\nproblem in computer vision and has many practical applications. Samples\ngenerated by existing text-to-image approaches can roughly reflect the meaning\nof the given descriptions, but they fail to contain necessary details and vivid\nobject parts. In this paper, we propose Stacked Generative Adversarial Networks\n(StackGAN) to generate 256x256 photo-realistic images conditioned on text\ndescriptions. We decompose the hard problem into more manageable sub-problems\nthrough a sketch-refinement process. The Stage-I GAN sketches the primitive\nshape and colors of the object based on the given text description, yielding\nStage-I low-resolution images. The Stage-II GAN takes Stage-I results and text\ndescriptions as inputs, and generates high-resolution images with\nphoto-realistic details. It is able to rectify defects in Stage-I results and\nadd compelling details with the refinement process. To improve the diversity of\nthe synthesized images and stabilize the training of the conditional-GAN, we\nintroduce a novel Conditioning Augmentation technique that encourages\nsmoothness in the latent conditioning manifold. Extensive experiments and\ncomparisons with state-of-the-arts on benchmark datasets demonstrate that the\nproposed method achieves significant improvements on generating photo-realistic\nimages conditioned on text descriptions.\n