Im2Vec: Synthesizing Vector Graphics without Vector Supervision

2021/02/04 by Pradyumna Reddy, Michaël Gharbi, Reddy, Pradyumna +7 · 1 voice · 27 citations
Computer Science · Engineering · Mathematics · #2D computer graphics #3D Shape Modeling and Analysis #3D computer graphics #Artificial intelligence #Computer Graphics and Visualization Techniques #Computer graphics #Computer graphics (images) #Computer science #Euclidean vector #Graphics #Image Processing and 3D Reconstruction #Mathematics #Raster graphics #Real-time computer graphics #Representation (politics) #Scalable Vector Graphics #Vector graphics #World Wide Web #cs.CV #cs.GR

paper · pdf · doi:10.48550/arxiv.2102.02798

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2021/02/04 · arxiv created 2021/04/01 · arxiv updated 2021/04/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Vector graphics are widely used to represent fonts, logos, digital artworks, and graphic designs. But, while a vast body of work has focused on generative algorithms for raster images, only a handful of options exists for vector graphics. One can always rasterize the input graphic and resort to image-based generative approaches, but this negates the advantages of the vector representation. The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time. This is not ideal because large-scale high quality vector-graphics datasets are difficult to obtain. Furthermore, the vector representation for a given design is not unique, so models that supervise on the vector representation are unnecessarily constrained. Instead, we propose a new neural network that can generate complex vector graphics with varying topologies, and only requires indirect supervision from readily-available raster training images (i.e., with no vector counterparts). To enable this, we use a differentiable rasterization pipeline that renders the generated vector shapes and composites them together onto a raster canvas. We demonstrate our method on a range of datasets, and provide comparison with state-of-the-art SVG-VAE and DeepSVG, both of which require explicit vector graphics supervision. Finally, we also demonstrate our approach on the MNIST dataset, for which no groundtruth vector representation is available. Source code, datasets, and more results are available at geometry.cs.ucl.ac.uk/projects/2021/Im2Vec/

Cited by

• VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing
• OmniSVG: A Unified Scalable Vector Graphics Generation Model
• VLMaterial: Procedural Material Generation with Large Vision-Language Models
• DOGE: Differentiable Bezier Graph Optimization for Road Network Extraction
• Illustrator's Depth: Monocular Layer Index Prediction for Image Decomposition
• SVG360: Editable Multiview Vector Graphics from a Single SVG
• Birth of a Painting: Differentiable Brushstroke Reconstruction
• Semantic Document Derendering: SVG Reconstruction via Vision-Language Modeling
• Stroke Modeling Enables Vectorized Character Generation with Large Vectorized Glyph Model
• SciTextures: Collecting and Connecting Visual Patterns, Models, and Code Across Science and Art
• From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration
• Vectorized Video Representation with Easy Editing via Hierarchical Spatio-Temporally Consistent Proxy Embedding
• Vector sketch animation generation with differentialable motion trajectories
• LayerD: Decomposing Raster Graphic Designs into Layers
• RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation
• An Exploratory Study on Abstract Images and Visual Representations Learned from Them
• Symbolic Graphics Programming with Large Language Models
• Structure-aware Contrastive Learning for Diagram Understanding of Multimodal Models
• Drawing2CAD: Sequence-to-Sequence Learning for CAD Generation from Vector Drawings
• SVGen: Interpretable Vector Graphics Generation with Large Language Models
• An Active Contour Model for Silhouette Vectorization using Bézier Curves
• Generative Learning of Differentiable Object Models for Compositional Interpretation of Complex Scenes
• SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation
• Reason-SVG: Enhancing Structured Reasoning for Vector Graphics Generation with Reinforcement Learning
• LayerPeeler: Autoregressive Peeling for Layer-wise Image Vectorization
• Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibility
• DesigNet: Learning to Draw Vector Graphics as Designers Do

Discussions

• Im2Vec: Synthesizing Vector Graphics Without Vector Supervision [hn, 91 points, 15 comments]

Related

• Image Vectorization: a Review
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation
• Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision
• DeepIcon: A Hierarchical Network for Layer-wise Icon Vectorization
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation