2021/12/20 by Nuriel Shalom Mor, Mor, Nuriel Shalom
Computer Science · Environmental Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #Image and Video Processing (eess.IV) #Remote Sensing and LiDAR Applications #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2112.11245
openalex publication_date 2021/12/20 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28
We examined the feasibility of generative adversarial networks (GANs) to generate photo-realistic images from LiDAR point clouds. For this purpose, we created a dataset of point cloud image pairs and trained the GAN to predict photorealistic images from LiDAR point clouds containing reflectance and distance information. Our models learned how to predict realistically looking images from just point cloud data, even images with black cars. Black cars are difficult to detect directly from point clouds because of their low level of reflectivity. This approach might be used in the future to perform visual object recognition on photorealistic images generated from LiDAR point clouds. In addition to the conventional LiDAR system, a second system that generates photorealistic images from LiDAR point clouds would run simultaneously for visual object recognition in real-time. In this way, we might preserve the supremacy of LiDAR and benefit from using photo-realistic images for visual object recognition without the usage of any camera. In addition, this approach could be used to colorize point clouds without the usage of any camera images.