Building LEGO Using Deep Generative Models of Graphs

2020/12/21 by Rylee Thompson, Thompson, Rylee, Elahe Ghalebi +5 · 1 citation
Computer Science · Engineering · #3D Shape Modeling and Analysis #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #Machine Learning (cs.LG) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2012.11543

NeurIPS 2020 ML4eng workshop paper

arxiv created 2020/12/21 · openalex publication_date 2020/12/21 · arxiv updated 2020/12/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Generative models are now used to create a variety of high-quality digital artifacts. Yet their use in designing physical objects has received far less attention. In this paper, we advocate for the construction toy, LEGO, as a platform for developing generative models of sequential assembly. We develop a generative model based on graph-structured neural networks that can learn from human-built structures and produce visually compelling designs. Our code is released at: https://github.com/uoguelph-mlrg/GenerativeLEGO.

Citations

• Kronecker Graphs: An Approach to Modeling Networks
• Learning Deep Generative Models of Graphs
• Syntax-Directed Variational Autoencoder for Structured Data
• Junction Tree Variational Autoencoder for Molecular Graph Generation
• Improved Precision and Recall Metric for Assessing Generative Models
• Reliable Fidelity and Diversity Metrics for Generative Models
• Language Models are Few-Shot Learners

Cited by

• Generating Physically Stable and Buildable Brick Structures from Text

Related

• Break and Make: Interactive Structural Understanding Using LEGO Bricks
• Learning to Build by Building Your Own Instructions
• Image2Lego: Customized LEGO Set Generation from Images
• TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly
• Translating a Visual LEGO Manual to a Machine-Executable Plan