2025/05/20 by Abhimanyu Talwar, Talwar, Abhimanyu, Julien Laasri +1
Earth and Planetary Sciences · #3D Surveying and Cultural Heritage #68T45 #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #I.2.10 #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2505.14621
openalex publication_date 2025/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider the problem of reconstructing a 3D scene from multiple sketches. We propose a pipeline which involves (1) stitching together multiple sketches through use of correspondence points, (2) converting the stitched sketch into a realistic image using a CycleGAN, and (3) estimating that image's depth-map using a pre-trained convolutional neural network based architecture called MegaDepth. Our contribution includes constructing a dataset of image-sketch pairs, the images for which are from the Zurich Building Database, and sketches have been generated by us. We use this dataset to train a CycleGAN for our pipeline's second step. We end up with a stitching process that does not generalize well to real drawings, but the rest of the pipeline that creates a 3D reconstruction from a single sketch performs quite well on a wide variety of drawings.