2021/08/19 by Kyle Lennon, Kyle R. Lennon, Katharina Fransen +16 · 1 voice · 4 citations
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Image and Video Retrieval Techniques #Advanced Vision and Imaging #Architecture #Artificial intelligence #Artificial neural network #Autoencoder #Computer graphics (images) #Computer science #Computer vision #Flexibility (engineering) #Matching (statistics) #Octree #Programming language #Representation (politics) #Set (abstract data type) #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2108.08477
published in arXiv (Cornell University) (Cornell University) · 9 pages, 10 figures
arxiv created 2021/08/19 · openalex publication_date 2021/08/19 · arxiv updated 2021/08/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Although LEGO sets have entertained generations of children and adults, the challenge of designing customized builds matching the complexity of real-world or imagined scenes remains too great for the average enthusiast. In order to make this feat possible, we implement a system that generates a LEGO brick model from 2D images. We design a novel solution to this problem that uses an octree-structured autoencoder trained on 3D voxelized models to obtain a feasible latent representation for model reconstruction, and a separate network trained to predict this latent representation from 2D images. LEGO models are obtained by algorithmic conversion of the 3D voxelized model to bricks. We demonstrate first-of-its-kind conversion of photographs to 3D LEGO models. An octree architecture enables the flexibility to produce multiple resolutions to best fit a user's creative vision or design needs. In order to demonstrate the broad applicability of our system, we generate step-by-step building instructions and animations for LEGO models of objects and human faces. Finally, we test these automatically generated LEGO sets by constructing physical builds using real LEGO bricks.