Occluded object reconstruction for first responders with augmented reality glasses using conditional generative adversarial networks

2018/04/20 by Kyongsik Yun, Thomas Lu, Yun, Kyongsik +3
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1805.00322

SPIE 2018

arxiv created 2018/04/20 · arxiv updated 2018/05/02

Abstract

Firefighters suffer a variety of life-threatening risks, including line-of-duty deaths, injuries, and exposures to hazardous substances. Support for reducing these risks is important. We built a partially occluded object reconstruction method on augmented reality glasses for first responders. We used a deep learning based on conditional generative adversarial networks to train associations between the various images of flammable and hazardous objects and their partially occluded counterparts. Our system then reconstructed an image of a new flammable object. Finally, the reconstructed image was superimposed on the input image to provide "transparency". The system imitates human learning about the laws of physics through experience by learning the shape of flammable objects and the flame characteristics.

Related

• Integrating Deep Learning and Augmented Reality to Enhance Situational Awareness in Firefighting Environments
• An embedded deep learning system for augmented reality in firefighting applications
• A Deep Learning Framework for Detection of Targets in Thermal Images to Improve Firefighting
• Predicting Rapid Fire Growth (Flashover) Using Conditional Generative Adversarial Networks
• Fire360: A Benchmark for Robust Perception and Episodic Memory in Degraded 360-Degree Firefighting Videos