Why ReLU networks yield high-confidence predictions far away from the\n training data and how to mitigate the problem

2018/12/13 by Matthias Hein, Maksym Andriushchenko, Hein, Matthias +3 · 19 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1812.05720

openalex publication_date 2018/12/13 · openalex created_date 2022/08/01 · openalex updated_date 2026/07/28

Abstract

Classifiers used in the wild, in particular for safety-critical systems,\nshould not only have good generalization properties but also should know when\nthey don't know, in particular make low confidence predictions far away from\nthe training data. We show that ReLU type neural networks which yield a\npiecewise linear classifier function fail in this regard as they produce almost\nalways high confidence predictions far away from the training data. For bounded\ndomains like images we propose a new robust optimization technique similar to\nadversarial training which enforces low confidence predictions far away from\nthe training data. We show that this technique is surprisingly effective in\nreducing the confidence of predictions far away from the training data while\nmaintaining high confidence predictions and test error on the original\nclassification task compared to standard training.\n

Cited by

• Am I Confused or Is This Confusing?: Deep Ensembles for ENSO Uncertainty Quantification
• Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection
• Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers
• Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
• Latent Uncertainty Representations for Video-based Driver Action and Intention Recognition
• Smoothing-Based Conformal Prediction for Balancing Efficiency and Interpretability
• Marching Neurons: Accurate Surface Extraction for Neural Implicit Shapes
• ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
• Identifying Adaptive Footprints in the Presence of Demographic Uncertainty
• Graph Enhanced Trajectory Anomaly Detection
• An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
• Contrastive Learning with Enhanced Abstract Representations using Grouped Loss of Abstract Semantic Supervision
• Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection
• Multi-Method Ensemble for Out-of-Distribution Detection
• Calibrated and uncertain? Evaluating uncertainty estimates in binary classification models
• Collision-based Watermark for Detecting Backdoor Manipulation in Federated Learning
• Uncertainty-Aware Likelihood Ratio Estimation for Pixel-Wise Out-of-Distribution Detection
• OoDDINO:A Multi-level Framework for Anomaly Segmentation on Complex Road Scenes
• Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality

Related

• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks
• Towards neural networks that provably know when they don't know
• An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their\n Asymptotic Overconfidence
• Overparameterized ReLU Neural Networks Learn the Simplest Models: Neural Isometry and Exact Recovery