Energy-based Out-of-distribution Detection
2020/10/08 by Weitang Liu, Liu, Weitang, Xiaoyun Wang +5 · 242 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.LG
paper · pdf · doi:10.48550/arxiv.2010.03759
openalex publication_date 2020/10/08 · arxiv created 2021/04/26 · arxiv updated 2021/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
Abstract
Determining whether inputs are out-of-distribution (OOD) is an essential building block for safely deploying machine learning models in the open world. However, previous methods relying on the softmax confidence score suffer from overconfident posterior distributions for OOD data. We propose a unified framework for OOD detection that uses an energy score. We show that energy scores better distinguish in- and out-of-distribution samples than the traditional approach using the softmax scores. Unlike softmax confidence scores, energy scores are theoretically aligned with the probability density of the inputs and are less susceptible to the overconfidence issue. Within this framework, energy can be flexibly used as a scoring function for any pre-trained neural classifier as well as a trainable cost function to shape the energy surface explicitly for OOD detection. On a CIFAR-10 pre-trained WideResNet, using the energy score reduces the average FPR (at TPR 95%) by 18.03% compared to the softmax confidence score. With energy-based training, our method outperforms the state-of-the-art on common benchmarks.
Citations
Cited by
- Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation
- Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
- Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis
- A New Kind of Adversarial Example: Measuring the Human-Model Gap, and Its Relationship to OOD Detection
- Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
- Out-of-Distribution Detection in Molecular Complexes via Diffusion Models for Irregular Graphs
- Coarse-to-Fine Open-Set Graph Node Classification with Large Language Models
- Out-of-Distribution Detection for Continual Learning: Design Principles and Benchmarking
- Interoceptive machine framework: Toward interoception-inspired regulatory architectures in artificial intelligence
- Privacy-Enhancing Infant Cry Classification with Federated Transformers and Denoising Regularization
- General OOD Detection via Model-aware and Subspace-aware Variable Priority
- Predictive Sample Assignment for Semantically Coherent Out-of-Distribution Detection
- Selective Conformal Risk Control
- Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring
- Representation Calibration and Uncertainty Guidance for Class-Incremental Learning based on Vision Language Model
- Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
- SPAD: Seven-Source Token Probability Attribution with Syntactic Aggregation for Detecting Hallucinations in RAG
- SPOOF: Simple Pixel Operations for Out-of-Distribution Fooling
- Domain Feature Collapse: Implications for Out-of-Distribution Detection and Solutions
- Credal Graph Neural Networks
- Calibration-Free EEG-based Driver Drowsiness Detection with Online Test-Time Adaptation
- TIE: A Training-Inversion-Exclusion Framework for Visually Interpretable and Uncertainty-Guided Out-of-Distribution Detection
- Continual Error Correction on Low-Resource Devices
- RankOOD -- Class Ranking-based Out-of-Distribution Detection
- SupLID: Geometrical Guidance for Out-of-Distribution Detection in Semantic Segmentation
- Robust Multimodal Sentiment Analysis of Image-Text Pairs by Distribution-Based Feature Recovery and Fusion
- SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data
- Exploiting Inter-Sample Information for Long-tailed Out-of-Distribution Detection
- ProtoAnomalyNCD: Prototype Learning for Multi-class Novel Anomaly Discovery in Industrial Scenarios
- TSRE: Channel-Aware Typical Set Refinement for Out-of-Distribution Detection
- Graph Memory: A Structured and Interpretable Framework for Modality-Agnostic Embedding-Based Inference
- Natural gradient descent for improving variational inference based classification of radio galaxies
- BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapse
- Calibrated Decomposition of Aleatoric and Epistemic Uncertainty in Deep Features for Inference-Time Adaptation
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts
- CATCHFed: Efficient Unlabeled Data Utilization for Semi-Supervised Federated Learning in Limited Labels Environments
- STORM: Segment, Track, and Object Re-Localization from a Single Image
- Benchmarking GNNs for OOD Materials Property Prediction with Uncertainty Quantification
- UCDSC: Open Set UnCertainty aware Deep Simplex Classifier for Medical Image Datasets
- Automatic Grid Updates for Kolmogorov-Arnold Networks using Layer Histograms
- ClusterMine: Robust Label-Free Visual Out-Of-Distribution Detection via Concept Mining from Text Corpora
- Relative Energy Learning for LiDAR Out-of-Distribution Detection
- Proto-LeakNet: Towards Signal-Leak Aware Attribution in Synthetic Human Face Imagery
- Uncertainty Guided Online Ensemble for Non-stationary Data Streams in Fusion Science
- Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection
- Covariance Last-Layer Ensembles: Function-Space Diversity for Efficient Uncertainty Quantification
- Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer
- Representation Trajectories Matters: Complementary Evidence for OOD Detection and Image Classification
- Improving Out-of-Distribution Detection via Dynamic Covariance Calibration
- Face-Trace: Open-Set Attribution and Progressive Discovery of Synthetic Face Generators
- Open-set recognition with long-tail sonar images
- When Rule Violations Are Rare: Chimera Training for Logical Anomaly Detection
- Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systems
- Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
- Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction
- Beyond Binary Out-of-Distribution Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectories
- GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection
- Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring
- Selective Labeling with False Discovery Rate Control
- Towards Adversarial Robustness and Uncertainty Quantification in DINOv2-based Few-Shot Anomaly Detection
- A Multi-dimensional Semantic Surprise Framework Based on Low-Entropy Semantic Manifolds for Fine-Grained Out-of-Distribution Detection
- SCOPED: Score-Curvature Out-of-distribution Proximity Evaluator for Diffusion
- Generation Space Size: Understanding and Calibrating Open-Endedness of LLM Generations
- Local Background Features Matter in Out-of-Distribution Detection
- ΔEnergy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalization
- A Machine Learning Perspective on Automated Driving Corner Cases
- Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection
- Uncovering Overconfident Failures in CXR Models via Augmentation-Sensitivity Risk Scoring
- Verifying Chain-of-Thought Reasoning via Its Computational Graph
- Hi-OSCAR: Hierarchical Open-set Classifier for Human Activity Recognition
- EigenScore: OOD Detection using Covariance in Diffusion Models
- The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
- Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection
- Knowledge Distillation Detection for Open-weights Models
- Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking
- Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimization
- Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
- Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy
- ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
- Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles
- PGTuner: An Efficient Framework for Automatic and Transferable Configuration Tuning of Proximity Graphs
- Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems
- Text Meets Topology: Rethinking Out-of-distribution Detection in Text-Rich Networks
- PRISM: Precision-Recall Informed Data-Free Knowledge Distillation via Generative Diffusion
- AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software
- Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning
- Quantifying Self-Awareness of Knowledge in Large Language Models
- Self-Guided Target Sound Extraction and Classification Through Universal Sound Separation Model and Multiple Clues
- Dynamic Aware: Adaptive Multi-Mode Out-of-Distribution Detection for Trajectory Prediction in Autonomous Vehicles
- An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
- Sharpness-Aware Geometric Defense for Robust Out-Of-Distribution Detection
- Road Obstacle Video Segmentation
- Optimizing Latent Dimension Allocation in Hierarchical VAEs: Balancing Attenuation and Information Retention for OOD Detection
- D2HScore: Reasoning-Aware Hallucination Detection via Semantic Breadth and Depth Analysis in LLMs
- Prompt Optimization Meets Subspace Representation Learning for Few-shot Out-of-Distribution Detection
- Tackling the Noisy Elephant in the Room: Label Noise-robust Out-of-Distribution Detection via Loss Correction and Low-rank Decomposition
- From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
- DCV-ROOD Evaluation Framework: Dual Cross-Validation for Robust Out-of-Distribution Detection
- Prior Distribution and Model Confidence
- Towards Open World Detection: A Survey
- ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
- Exploring Vicinal Risk Minimization for Lightweight Out-of-Distribution Detection
- RS-OOD: A Vision-Language Augmented Framework for Out-of-Distribution Detection in Remote Sensing
- Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection
- Energy Landscapes Enable Reliable Abstention in Retrieval-Augmented Large Language Models for Healthcare
- Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains
- Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks
- Activation Subspaces for Out-of-Distribution Detection
- Multi-Method Ensemble for Out-of-Distribution Detection
- Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detection
- Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
- Random forest-based out-of-distribution detection for robust lung cancer segmentation
- Semantic Energy: Detecting LLM Hallucination Beyond Entropy
- ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification
- SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
- Retrieval-Augmented Prompt for OOD Detection
- Out-of-Distribution Detection using Counterfactual Distance
- OpenHAIV: A Framework Towards Practical Open-World Learning
- Generalized Few-Shot Out-of-Distribution Detection
- Dual Prompt Learning for Adapting Vision-Language Models to Downstream Image-Text Retrieval
- Pseudo-label Induced Subspace Representation Learning for Robust Out-of-Distribution Detection
- TRACEALIGN -- Tracing the Drift: Attributing Alignment Failures to Training-Time Belief Sources in LLMs
- COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition
- A Simple and Effective Method for Uncertainty Quantification and OOD Detection
- BOOD: Boundary-based Out-Of-Distribution Data Generation
- Model Predictive Adversarial Imitation Learning for Planning from Observation
- Knowledge Regularized Negative Feature Tuning of Vision-Language Models for Out-of-Distribution Detection
- Bag of Coins: A Statistical Probe into Neural Confidence Structures
- Handling Out-of-Distribution Data: A Survey
- Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
- Characterizing Linear Alignment Across Language Models
- Robust Explanations Through Uncertainty Decomposition: A Path to Trustworthier AI
- LanePerf: a Performance Estimation Framework for Lane Detection
- Inherently Faithful Attention Maps for Vision Transformers
- Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- Test-Time Canonicalization by Foundation Models for Robust Perception
- NegRefine: Refining Negative Label-Based Zero-Shot OOD Detection
- Monitoring Risks in Test-Time Adaptation
- Feature Bank Enhancement for Distance-based Out-of-Distribution Detection
- Elsa: Energy-based learning for semi-supervised anomaly detection
- DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging
- Energy-Guided Decoding for Object Hallucination Mitigation
- Open-Set Source Tracing of Audio Deepfake Systems
- FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection
- When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
- Understanding Knowledge Transferability for Transfer Learning: A Survey
- Gradient Short-Circuit: Efficient Out-of-Distribution Detection via Feature Intervention
- Out-of-Distribution Detection Methods Answer the Wrong Questions
- Out-of-Distribution Detection with Adaptive Top-K Logits Integration
- A New Perspective On AI Safety Through Control Theory Methodologies
- Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detection
- Exploiting Vision Language Model for Training-Free 3D Point Cloud OOD Detection via Graph Score Propagation
- SODA: Out-of-Distribution Detection in Domain-Shifted Point Clouds via Neighborhood Propagation
- Evidential Spectrum-Aware Contrastive Learning for OOD Detection in Dynamic Graphs
- HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations
- Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detection
- Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation
- Energy-Based Transfer for Reinforcement Learning
- Reliable Few-shot Learning under Dual Noises
- TRUST: Transparent, Robust and Ultra-Sparse Trees
- NERO: Explainable Out-of-Distribution Detection with Neuron-level Relevance
- Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations
- A Variational Information Theoretic Approach to Out-of-Distribution Detection
- EVINET: Towards Open-World Graph Learning via Evidential Reasoning Network
- FindMeIfYouCan: Bringing Open Set metrics to near , far and farther Out-of-Distribution Object Detection
- Learning to Fuse: Modality-Aware Adaptive Scheduling for Robust Multimodal Foundation Models
- Balanced Hyperbolic Embeddings Are Natural Out-of-Distribution Detectors
- Improving Wildlife Out-of-Distribution Detection: Africas Big Five
- Enhancing Abnormality Identification: Robust Out-of-Distribution Strategies for Deepfake Detection
- Multi-Modal Dataset Distillation in the Wild
- Position: Epistemic uncertainty estimation methods are fundamentally incomplete
- Buffer-free Class-Incremental Learning with Out-of-Distribution Detection
- To Trust Or Not To Trust Your Vision-Language Model's Prediction
- Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Loss
- TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models
- Improving Out-of-Distribution Detection with Markov Logic Networks
- Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory
- AquaMonitor: A multimodal multi-view image sequence dataset for real-life aquatic invertebrate biodiversity monitoring
- Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layer
- ASAT: Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection
- Recent Advances in Out-of-Distribution Detection with CLIP-Like Models: A Survey
- Energy-based Preference Optimization for Test-time Adaptation
- Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression
- Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering
- Test-Time Adaptation with Binary Feedback
- Mahalanobis++: Improving OOD Detection via Feature Normalization
- ReFP-AD: Rectified Flow Preconditioning for Energy-Based Anomaly Detection
- Boosting Open Set Recognition Performance through Modulated Representation Learning
- Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
- TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbation
- GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection
- Refining Neural Activation Patterns for Layer-Level Concept Discovery in Neural Network-Based Receivers
- Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selections and Approximations
- SpectralGap: Graph-Level Out-of-Distribution Detection via Laplacian Eigenvalue Gaps
- RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detection
- Learning to Rank Chain-of-Thought: Using a Small Model
- Know When to Abstain: Optimal Selective Classification with Likelihood Ratios
- Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift
- When to retrain a machine learning model
- LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Data
- GEM: Gaussian Embedding Modeling for Out-of-Distribution Detection in GUI Agents
- Training-Free Out-of-Distribution Detection for Pathology Whole-Slide Images
- Natural Attribute-based Shift Detection
- Open Set Domain Adaptation with Vision-language models via Gradient-aware Separation
- Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensembles
- Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
- Response Magnitude as a Dominant Signal for Held-Out CRISPRi Perturbation Effect Prediction
- No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets
- Unsupervised Out-of-Distribution Detection in Medical Imaging Using Multi-Exit Class Activation Maps and Feature Masking
- Open Set Label Shift with Test Time Out-of-Distribution Reference
- Incorporating Inductive Biases to Energy-based Generative Models
- Utilising Gradient-Based Proposals Within Sequential Monte Carlo Samplers for Training of Partial Bayesian Neural Networks
- Energy Aligning for Biased Models
- Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
- Spilled Energy in Large Language Models
- Self-Aware Object Detection via Degradation Manifolds
- A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift
- Graph Synthetic Out-of-Distribution Exposure with Large Language Models
- Out-of-Distribution Detection in Heterogeneous Graphs via Energy Propagation
- GLIP-OOD: Zero-Shot Graph OOD Detection with Graph Foundation Model
- Open-set Anomaly Segmentation in Complex Scenarios
- Hallucination in World Models is Predictable and Preventable
- Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection
- Enhancing System Self-Awareness and Trust of AI: A Case Study in Trajectory Prediction and Planning
- EPSILON: Adaptive Fault Mitigation in Approximate Deep Neural Network using Statistical Signatures
- SABRE: A Multi-Agent Approach for Selecting Out-of-Distribution Detectors Under a Budget
- Explaining Low Perception Model Competency with High-Competency Counterfactuals
- Can We Ignore Labels In Out of Distribution Detection?
- TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data
- Bounded and Uniform Energy-based Out-of-distribution Detection for Graphs
- Energy-Based Reward Models for Robust Language Model Alignment
- Maximizing Discrimination Capability of Knowledge Distillation with Energy Function
- Uncertainty-Aware World Model for Aerial Image-Goal Navigation
- Enhancing Out-of-Distribution Detection with Extended Logit Normalization
- Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery
- Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations
- Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation
- EOOD: Entropy-based Out-of-distribution Detection
- STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainability
Related