nn-dependability-kit: Engineering Neural Networks for Safety-Critical Autonomous Driving Systems

2018/11/16 by Chih‐Hong Cheng, Cheng, Chih-Hong, Chung-Hao Huang +3 · 1 citation
Computer Science · Decision Sciences · Engineering · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Risk and Safety Analysis #Safety Systems Engineering in Autonomy

paper · pdf · doi:10.48550/arxiv.1811.06746

openalex publication_date 2018/11/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Can engineering neural networks be approached in a disciplined way similar to how engineers build software for civil aircraft? We present nn-dependability-kit, an open-source toolbox to support safety engineering of neural networks for autonomous driving systems. The rationale behind nn-dependability-kit is to consider a structured approach (via Goal Structuring Notation) to argue the quality of neural networks. In particular, the tool realizes recent scientific results including (a) novel dependability metrics for indicating sufficient elimination of uncertainties in the product life cycle, (b) formal reasoning engine for ensuring that the generalization does not lead to undesired behaviors, and (c) runtime monitoring for reasoning whether a decision of a neural network in operation is supported by prior similarities in the training data. A proprietary version of nn-dependability-kit has been used to improve the quality of a level-3 autonomous driving component developed by Audi for highway maneuvers.

Citations

• End to End Learning for Self-Driving Cars
• A Dual Approach to Scalable Verification of Deep Networks
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Cited by

• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability

Related

• Towards Dependability Metrics for Neural Networks
• Neural Networks for Safety-Critical Applications - Challenges, Experiments and Perspectives
• Hardening of Artificial Neural Networks for Use in Safety-Critical Applications -- A Mapping Study
• Trusted Neural Networks for Safety-Constrained Autonomous Control
• Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry