2021/02/24 by Osama Mazhar, Mazhar, Osama, Robert Babuška +4 · 1 citation
Computer Science · Engineering · Environmental Science · #Advanced Chemical Sensor Technologies #Advanced Neural Network Applications #Air Quality Monitoring and Forecasting #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #End-to-end principle #FOS: Computer and information sciences #Object detection #Pattern recognition (psychology) #RGB color model #Real-time computing #Redundancy (engineering) #Robot #Robotics (cs.RO) #Robustness (evolution) #Sensor fusion #cs.CV #cs.RO
paper · pdf · doi:10.48550/arxiv.2102.12319
published in arXiv (Cornell University) (Cornell University) · IEEE Robotics and Automation Letters (RA-L)
openalex publication_date 2021/02/24 · arxiv created 2021/06/22 · arxiv updated 2021/06/23 · openalex created_date 2022/07/25 · openalex updated_date 2026/08/05
Deep neural networks designed for vision tasks are often prone to failure\nwhen they encounter environmental conditions not covered by the training data.\nSingle-modal strategies are insufficient when the sensor fails to acquire\ninformation due to malfunction or its design limitations. Multi-sensor\nconfigurations are known to provide redundancy, increase reliability, and are\ncrucial in achieving robustness against asymmetric sensor failures. To address\nthe issue of changing lighting conditions and asymmetric sensor degradation in\nobject detection, we develop a multi-modal 2D object detector, and propose\ndeterministic and stochastic sensor-aware feature fusion strategies. The\nproposed fusion mechanisms are driven by the estimated sensor measurement\nreliability values/weights. Reliable object detection in harsh lighting\nconditions is essential for applications such as self-driving vehicles and\nhuman-robot interaction. We also propose a new "r-blended" hybrid depth\nmodality for RGB-D sensors. Through extensive experimentation, we show that the\nproposed strategies outperform the existing state-of-the-art methods on the\nFLIR-Thermal dataset, and obtain promising results on the SUNRGB-D dataset. We\nadditionally record a new RGB-Infra indoor dataset, namely L515-Indoors, and\ndemonstrate that the proposed object detection methodologies are highly\neffective for a variety of lighting conditions.\n