Logit-based Uncertainty Measure in Classification

2021/07/06 by Huiyu Wu, Wu, Huiyu, Diego Klabjan +1 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #cs.LG

paper · pdf · doi:10.48550/arxiv.2107.02845

arxiv created 2021/07/06 · openalex publication_date 2021/07/06 · arxiv updated 2021/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a new, reliable, and agnostic uncertainty measure for classification tasks called logit uncertainty. It is based on logit outputs of neural networks. We in particular show that this new uncertainty measure yields a superior performance compared to existing uncertainty measures on different tasks, including out of sample detection and finding erroneous predictions. We analyze theoretical foundations of the measure and explore a relationship with high density regions. We also demonstrate how to test uncertainty using intermediate outputs in training of generative adversarial networks. We propose two potential ways to utilize logit-based uncertainty in real world applications, and show that the uncertainty measure outperforms.

Citations

• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• To Trust Or Not To Trust A Classifier
• Weight Uncertainty in Neural Networks
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Cited by

• Are you sure? Measuring models bias in content moderation through uncertainty

Related

• A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers
• Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications
• Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
• Measuring Orthogonality as the Blind-Spot of Uncertainty Disentanglement
• Nonparametric Uncertainty Quantification for Single Deterministic Neural Network