Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction

2019/12/04 by Fuyang Zhang, Zhang, Fuyang, Nelson Nauata +3 · 5 citations
Computer Science · Environmental Science · #Advanced Vision and Imaging #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Convolutional neural network #Embedding #FOS: Computer and information sciences #Graph #Pattern recognition (psychology) #Remote Sensing and LiDAR Applications #Theoretical computer science #Visual Attention and Saliency Detection #cs.CV

paper · pdf · doi:10.48550/arxiv.1912.01756

published in arXiv (Cornell University) (Cornell University) · Accepted by CVPR2020

openalex publication_date 2019/12/04 · arxiv created 2021/06/07 · arxiv updated 2021/06/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

This paper proposes a novel message passing neural (MPN) architecture Conv-MPN, which reconstructs an outdoor building as a planar graph from a single RGB image. Conv-MPN is specifically designed for cases where nodes of a graph have explicit spatial embedding. In our problem, nodes correspond to building edges in an image. Conv-MPN is different from MPN in that 1) the feature associated with a node is represented as a feature volume instead of a 1D vector; and 2) convolutions encode messages instead of fully connected layers. Conv-MPN learns to select a true subset of nodes (i.e., building edges) to reconstruct a building planar graph. Our qualitative and quantitative evaluations over 2,000 buildings show that Conv-MPN makes significant improvements over the existing fully neural solutions. We believe that the paper has a potential to open a new line of graph neural network research for structured geometry reconstruction.

Citations

• Spectral Networks and Locally Connected Networks on Graphs
• ImageNet Large Scale Visual Recognition Challenge
• Semi-Supervised Classification with Graph Convolutional Networks
• Neural Message Passing for Quantum Chemistry
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal\n Networks
• SpaceNet: A Remote Sensing Dataset and Challenge Series
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Related

• Vectorizing World Buildings: Planar Graph Reconstruction by Primitive Detection and Relationship Inference
• Optimized Vectorizing of Building Structures with Switch: High-Efficiency Convolutional Channel-Switch Hybridization Strategy
• Building Segmentation through a Gated Graph Convolutional Neural Network with Deep Structured Feature Embedding
• Geom-GCN: Geometric Graph Convolutional Networks
• FloorNet: A Unified Framework for Floorplan Reconstruction from 3D Scans