Understanding Softmax Confidence and Uncertainty

2021/06/09 by Tim Pearce, Alexandra Brintrup, Pearce, Tim +3 · 8 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.AI #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2106.04972

arxiv created 2021/06/09 · openalex publication_date 2021/06/09 · arxiv updated 2021/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution. Yet naively using softmax confidence as a proxy for uncertainty achieves modest success in tasks exclusively testing for this, e.g., out-of-distribution (OOD) detection. This paper investigates this contradiction, identifying two implicit biases that do encourage softmax confidence to correlate with epistemic uncertainty: 1) Approximately optimal decision boundary structure, and 2) Filtering effects of deep networks. It describes why low-dimensional intuitions about softmax confidence are misleading. Diagnostic experiments quantify reasons softmax confidence can fail, finding that extrapolations are less to blame than overlap between training and OOD data in final-layer representations. Pre-trained/fine-tuned networks reduce this overlap.

Citations

• A Practical Bayesian Framework for Backpropagation Networks
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Cited by

• Uncertainty-Aware Subset Selection for Robust Visual Explainability under Distribution Shifts
• Confidence Driven Classification of Application Types in the Presence of Background Network Traffic
• The Confidence Paradox: Can LLM Know When It's Wrong
• Pieceformer: Similarity-Driven Knowledge Transfer via Scalable Graph Transformer in VLSI
• TRUST: Test-time Resource Utilization for Superior Trustworthiness
• Recursive Offloading for LLM Serving in Multi-tier Networks
• Real-World fNIRS-Based Brain-Computer Interfaces: Benchmarking Deep Learning and Classical Models in Interactive Gaming
• OODTE: A Differential Testing Engine for the ONNX Optimizer

Related

• Explorations of the Softmax Space: Knowing When the Neural Network Doesn't Know
• Inhibited Softmax for Uncertainty Estimation in Neural Networks
• Energy-Based Open-World Uncertainty Modeling for Confidence Calibration
• Fine-grained Uncertainty Modeling in Neural Networks
• Softmax Is Not an Artificial Trick: An Information-Theoretic View of Softmax in Neural Networks