2021/07/06 by Takami Sato, Qi Alfred Chen, Sato, Takami +1
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Adversarial system #Anomaly Detection Techniques and Applications #Artificial intelligence #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Cryptography and Security (cs.CR) #De facto #FOS: Computer and information sciences #Machine learning #Robustness (evolution) #Software deployment #cs.CR #cs.CV
paper · pdf · doi:10.48550/arxiv.2107.02488
arxiv created 2021/07/06 · openalex publication_date 2021/07/06 · arxiv updated 2021/07/07 · openalex created_date 2021/07/19 · openalex updated_date 2026/07/28
After the 2017 TuSimple Lane Detection Challenge, its evaluation based on accuracy and F1 score has become the de facto standard to measure the performance of lane detection methods. In this work, we conduct the first large-scale empirical study to evaluate the robustness of state-of-the-art lane detection methods under physical-world adversarial attacks in autonomous driving. We evaluate 4 major types of lane detection approaches with the conventional evaluation and end-to-end evaluation in autonomous driving scenarios and then discuss the security proprieties of each lane detection model. We demonstrate that the conventional evaluation fails to reflect the robustness in end-to-end autonomous driving scenarios. Our results show that the most robust model on the conventional metrics is the least robust in the end-to-end evaluation. Although the competition dataset and its metrics have played a substantial role in developing performant lane detection methods along with the rapid development of deep neural networks, the conventional evaluation is becoming obsolete and the gap between the metrics and practicality is critical. We hope that our study will help the community make further progress in building a more comprehensive framework to evaluate lane detection models.