Energy-based Out-of-distribution Detection
2020/10/08 by Weitang Liu, Liu, Weitang, Xiaoyun Wang +5 · 130 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2010.03759
openalex publication_date 2020/10/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
Abstract
Determining whether inputs are out-of-distribution (OOD) is an essential building block for safely deploying machine learning models in the open world. However, previous methods relying on the softmax confidence score suffer from overconfident posterior distributions for OOD data. We propose a unified framework for OOD detection that uses an energy score. We show that energy scores better distinguish in- and out-of-distribution samples than the traditional approach using the softmax scores. Unlike softmax confidence scores, energy scores are theoretically aligned with the probability density of the inputs and are less susceptible to the overconfidence issue. Within this framework, energy can be flexibly used as a scoring function for any pre-trained neural classifier as well as a trainable cost function to shape the energy surface explicitly for OOD detection. On a CIFAR-10 pre-trained WideResNet, using the energy score reduces the average FPR (at TPR 95%) by 18.03% compared to the softmax confidence score. With energy-based training, our method outperforms the state-of-the-art on common benchmarks.
Citations
Cited by
- Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation
- Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
- Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis
- A New Kind of Adversarial Example: Measuring the Human-Model Gap, and Its Relationship to OOD Detection
- Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
- Out-of-Distribution Detection in Molecular Complexes via Diffusion Models for Irregular Graphs
- Coarse-to-Fine Open-Set Graph Node Classification with Large Language Models
- Out-of-Distribution Detection for Continual Learning: Design Principles and Benchmarking
- Interoceptive machine framework: Toward interoception-inspired regulatory architectures in artificial intelligence
- Privacy-Enhancing Infant Cry Classification with Federated Transformers and Denoising Regularization
- General OOD Detection via Model-aware and Subspace-aware Variable Priority
- Predictive Sample Assignment for Semantically Coherent Out-of-Distribution Detection
- Selective Conformal Risk Control
- Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring
- Representation Calibration and Uncertainty Guidance for Class-Incremental Learning based on Vision Language Model
- Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
- SPAD: Seven-Source Token Probability Attribution with Syntactic Aggregation for Detecting Hallucinations in RAG
- SPOOF: Simple Pixel Operations for Out-of-Distribution Fooling
- Domain Feature Collapse: Implications for Out-of-Distribution Detection and Solutions
- Credal Graph Neural Networks
- Calibration-Free EEG-based Driver Drowsiness Detection with Online Test-Time Adaptation
- TIE: A Training-Inversion-Exclusion Framework for Visually Interpretable and Uncertainty-Guided Out-of-Distribution Detection
- Continual Error Correction on Low-Resource Devices
- RankOOD -- Class Ranking-based Out-of-Distribution Detection
- SupLID: Geometrical Guidance for Out-of-Distribution Detection in Semantic Segmentation
- Robust Multimodal Sentiment Analysis of Image-Text Pairs by Distribution-Based Feature Recovery and Fusion
- SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data
- Exploiting Inter-Sample Information for Long-tailed Out-of-Distribution Detection
- ProtoAnomalyNCD: Prototype Learning for Multi-class Novel Anomaly Discovery in Industrial Scenarios
- TSRE: Channel-Aware Typical Set Refinement for Out-of-Distribution Detection
- Knowledge Graphs as Structured Memory for Embedding Spaces: From Training Clusters to Explainable Inference
- Natural gradient descent for improving variational inference based classification of radio galaxies
- BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapse
- Calibrated Decomposition of Aleatoric and Epistemic Uncertainty in Deep Features for Inference-Time Adaptation
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts
- CATCHFed: Efficient Unlabeled Data Utilization for Semi-Supervised Federated Learning in Limited Labels Environments
- STORM: Segment, Track, and Object Re-Localization from a Single Image
- Benchmarking GNNs for OOD Materials Property Prediction with Uncertainty Quantification
- UCDSC: Open Set UnCertainty aware Deep Simplex Classifier for Medical Image Datasets
- Automatic Grid Updates for Kolmogorov-Arnold Networks using Layer Histograms
- ClusterMine: Robust Label-Free Visual Out-Of-Distribution Detection via Concept Mining from Text Corpora
- Relative Energy Learning for LiDAR Out-of-Distribution Detection
- Proto-LeakNet: Towards Signal-Leak Aware Attribution in Synthetic Human Face Imagery
- Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Science
- Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection
- Covariance Last-Layer Ensembles: Function-Space Diversity for Efficient Uncertainty Quantification
- Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer
- Representation Trajectories Matters: Complementary Evidence for OOD Detection and Image Classification
- Face-Trace: Open-Set Attribution and Progressive Discovery of Synthetic Face Generators
- Open-set recognition with long-tail sonar images
- When Rule Violations Are Rare: Chimera Training for Logical Anomaly Detection
- Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems
- Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
- Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction
- Beyond Binary Out-of-Distribution Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectories
- GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection
- Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring
- Selective Labeling with False Discovery Rate Control
- Towards Adversarial Robustness and Uncertainty Quantification in DINOv2-based Few-Shot Anomaly Detection
- A Multi-dimensional Semantic Surprise Framework Based on Low-Entropy Semantic Manifolds for Fine-Grained Out-of-Distribution Detection
- SCOPED: Score-Curvature Out-of-distribution Proximity Evaluator for Diffusion
- Generation Space Size: Understanding and Calibrating Open-Endedness of LLM Generations
- Local Background Features Matter in Out-of-Distribution Detection
- ΔEnergy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalization
- A Machine Learning Perspective on Automated Driving Corner Cases
- Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection
- Uncovering Overconfident Failures in CXR Models via Augmentation-Sensitivity Risk Scoring
- Verifying Chain-of-Thought Reasoning via Its Computational Graph
- Hi-OSCAR: Hierarchical Open-set Classifier for Human Activity Recognition
- EigenScore: OOD Detection using Covariance in Diffusion Models
- The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
- Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection
- Knowledge Distillation Detection for Open-weights Models
- Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking
- Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimization
- Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic\n Uncertainty
- Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy
- ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
- Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles
- PGTuner: An Efficient Framework for Automatic and Transferable Configuration Tuning of Proximity Graphs
- Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems
- Text Meets Topology: Rethinking Out-of-distribution Detection in Text-Rich Networks
- PRISM: Precision-Recall Informed Data-Free Knowledge Distillation via Generative Diffusion
- AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software
- Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning
- Quantifying Self-Awareness of Knowledge in Large Language Models
- Self-Guided Target Sound Extraction and Classification Through Universal Sound Separation Model and Multiple Clues
- Dynamic Aware: Adaptive Multi-Mode Out-of-Distribution Detection for Trajectory Prediction in Autonomous Vehicles
- An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
- Sharpness-Aware Geometric Defense for Robust Out-Of-Distribution Detection
- Road Obstacle Video Segmentation
- D2HScore: Reasoning-Aware Hallucination Detection via Semantic Breadth and Depth Analysis in LLMs
- Prompt Optimization Meets Subspace Representation Learning for Few-shot Out-of-Distribution Detection
- Tackling the Noisy Elephant in the Room: Label Noise-robust Out-of-Distribution Detection via Loss Correction and Low-rank Decomposition
- From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
- DCV-ROOD Evaluation Framework: Dual Cross-Validation for Robust Out-of-Distribution Detection
- Prior Distribution and Model Confidence
- Towards Open World Detection: A Survey
- ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
- Exploring Vicinal Risk Minimization for Lightweight Out-of-Distribution Detection
- RS-OOD: A Vision-Language Augmented Framework for Out-of-Distribution Detection in Remote Sensing
- Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection
- Energy Landscapes Enable Reliable Abstention in Retrieval-Augmented Large Language Models for Healthcare
- Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains
- Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks
- Activation Subspaces for Out-of-Distribution Detection
- Multi-Method Ensemble for Out-of-Distribution Detection
- Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detection
- Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
- Random forest-based out-of-distribution detection for robust lung cancer segmentation
- Semantic Energy: Detecting LLM Hallucination Beyond Entropy
- ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification
- SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
- Retrieval-Augmented Prompt for OOD Detection
- Out-of-Distribution Detection using Counterfactual Distance
- OpenHAIV: A Framework Towards Practical Open-World Learning
- Generalized Few-Shot Out-of-Distribution Detection
- Dual Prompt Learning for Adapting Vision-Language Models to Downstream Image-Text Retrieval
- Pseudo-label Induced Subspace Representation Learning for Robust Out-of-Distribution Detection
- TRACEALIGN -- Tracing the Drift: Attributing Alignment Failures to Training-Time Belief Sources in LLMs
- COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition
- A Simple and Effective Method for Uncertainty Quantification and OOD Detection
- BOOD: Boundary-based Out-Of-Distribution Data Generation
- Model Predictive Adversarial Imitation Learning for Planning from Observation
- Knowledge Regularized Negative Feature Tuning of Vision-Language Models for Out-of-Distribution Detection
- Bag of Coins: A Statistical Probe into Neural Confidence Structures
- Handling Out-of-Distribution Data: A Survey
- Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
Related