A Unifying Review of Deep and Shallow Anomaly Detection

2020/09/30 by Lukas Ruff, Jacob R. Kauffmann, Robert A. Vandermeulen +7 · 843 citations
Computer Science · Mathematics · #Anomaly Detection Techniques and Applications #Anomaly detection #Deep learning #Explainable Artificial Intelligence (XAI) #Field (mathematics) #Generative grammar #Imbalanced Data Classification Techniques #Relation (database) #Variety (cybernetics) #cs.AI #cs.LG #stat.ML

paper · pdf · open access · doi:10.1109/jproc.2021.3052449

published in Proceedings of the IEEE 109(5), 756-795 (Institute of Electrical and Electronics Engineers) · 40 pages; accepted for publication in the Proceedings of the IEEE;

openalex created_date 2020/10/01 · openalex publication_date 2021/02/05 · arxiv created 2021/02/08 · arxiv updated 2021/02/09 · openalex updated_date 2026/08/05

Abstract

Deep learning approaches to anomaly detection have recently improved the state of the art in detection performance on complex datasets such as large collections of images or text. These results have sparked a renewed interest in the anomaly detection problem and led to the introduction of a great variety of new methods. With the emergence of numerous such methods, including approaches based on generative models, one-class classification, and reconstruction, there is a growing need to bring methods of this field into a systematic and unified perspective. In this review we aim to identify the common underlying principles as well as the assumptions that are often made implicitly by various methods. In particular, we draw connections between classic 'shallow' and novel deep approaches and show how this relation might cross-fertilize or extend both directions. We further provide an empirical assessment of major existing methods that is enriched by the use of recent explainability techniques, and present specific worked-through examples together with practical advice. Finally, we outline critical open challenges and identify specific paths for future research in anomaly detection.

Citations

• Remarks on Some Nonparametric Estimates of a Density Function
• Analysis of a complex of statistical variables into principal components.
• A Survey of Outlier Detection Methods in Network Anomaly Identification
• On Estimation of a Probability Density Function and Mode
• Robust principal component analysis?
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Quantum-chemical insights from deep tensor neural networks

Cited by

• PriorProof: A Point-in-Time Measure of Technique Novelty for Formal Proofs
• Group Equivariant Diffusion for Anomaly Detection in Computational Cytology
• RcAE: Recursive Reconstruction Framework for Unsupervised Industrial Anomaly Detection
• Self-supervised anomaly detection in computer vision and beyond: A survey and outlook
• Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis
• Open-Set Domain Adaptation Under Background Distribution Shift: Challenges and A Provably Efficient Solution
• Generalized Out-of-Distribution Detection: A Survey
• Algorithmic detection of false data injection attacks in cyber-physical systems
• Explainable Deep Convolutional Multi-Type Anomaly Detection
• Recent advances in wind turbine condition monitoring using SCADA data: A state-of-the-art review
• Noise & pattern: identity-anchored Tikhonov regularization for robust structural anomaly detection
• Early Failure Prediction from Near-Anomaly Detection: A Proactive Approach
• Anomaly detection for high-content image-based phenotypic cell profiling
• Adaptable and Interpretable Framework for Anomaly Detection in SCADA-based industrial systems
• Cascaded Outlier-Aware Anomaly Detection for Cloud-Native Containers Under Extreme Class Imbalance
• When Rule Violations Are Rare: Chimera Training for Logical Anomaly Detection
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• INQMAD: incremental streaming anomaly detection with density matrices, quantum measurement, and density estimation
• An Evidence-Based Post-Hoc Adjustment Framework for Anomaly Detection Under Data Contamination
• ReTabAD: A Benchmark for Restoring Semantic Context in Tabular Anomaly Detection
• Failure Prediction at Runtime for Generative Robot Policies
• On Uniformly Scaling Flows: A Density-Aligned Approach to Deep One-Class Classification
• Out-of-Distribution Detection from Small Training Sets using Bayesian Neural Network Classifiers
• Anomaly-Aware YOLO: A Frugal yet Robust Approach to Infrared Small Target Detection
• LLM as an Algorithmist: Enhancing Anomaly Detectors via Programmatic Synthesis
• RDD: Pareto Analysis of the Rate-Distortion-Distinguishability Trade-off
• HyCoVAD: A Hybrid SSL-LLM Model for Complex Video Anomaly Detection
• Robust Anomaly Detection in Industrial Environments via Meta-Learning
• Introducing Resizable Region Packing Problem in Image Generation, with a Heuristic Solution
• Goal-Oriented Joint Source-Channel Coding: Distortion-Classification-Power Trade-off
• H-Alpha Anomalyzer: An Explainable Anomaly Detector for Solar H-Alpha Observations
• Scalable state change detection for stimulus-aware digital twins
• Contextual Learning for Anomaly Detection in Tabular Data
• RoseCDL: Robust and Scalable Convolutional Dictionary Learning for Rare event and Anomaly Detection
• PMODE: Theoretically Grounded and Modular Mixture Modeling
• One-Class Intrusion Detection with Dynamic Graphs
• Exploring Self-Supervised Audio Models for Generalized Anomalous Sound Detection
• Out-of-Distribution Detection using Counterfactual Distance
• Leveraging the Christoffel Function for Outlier Detection in Data Streams
• Rare anomalies require large datasets: About proving the existence of anomalies
• Leveraging large language models for SQL behavior-based database intrusion detection
• Agentic AI for autonomous anomaly management in complex systems
• SDVDiag: A Modular Platform for the Diagnosis of Connected Vehicle Functions
• OCSVM-Guided Representation Learning for Unsupervised Anomaly Detection
• Explainable AI guided unsupervised fault diagnostics for high-voltage circuit breakers
• The AIDE Toolbox: Artificial intelligence for disentangling extreme events [Software and Data Sets]
• Deep Learning With Noisy Labels for Spatiotemporal Drought Detection
• Text-ADBench: Text Anomaly Detection Benchmark based on LLMs Embedding
• Deep Anomaly Detection on Tennessee Eastman Process Data
• Towards Generalized Range-View LiDAR Segmentation in Adverse Weather
• UniOD: A Universal Model for Outlier Detection across Diverse Domains
• Out-of-Distribution Detection Methods Answer the Wrong Questions
• Bridging Unsupervised and Semi-Supervised Anomaly Detection: A Theoretically-Grounded and Practical Framework with Synthetic Anomalies
• Polyra Swarms: A Shape-Based Approach to Machine Learning
• Federated deep Q-learning networks for service-based anomaly detection and classification in edge-to-cloud ecosystems
• Using anomaly detection to search for technosignatures in Breakthrough Listen observations
• Investigating Mask-aware Prototype Learning for Tabular Anomaly Detection
• Continual-MEGA: A Large-scale Benchmark for Generalizable Continual Anomaly Detection
• MADCluster: Model-agnostic Anomaly Detection with Self-supervised Clustering Network
• NOVA: A Benchmark for Anomaly Localization and Clinical Reasoning in Brain MRI
• Unsupervised anomalous sound detection for industrial monitoring based on ArcFace classifier and gaussian mixture model
• Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
• ArcAD: Anomaly-Rectified Calibration for Cold-Start Supervised Anomaly Detection
• Quantum Support Vector Regression for Robust Anomaly Detection
• Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
• Handcrafted Feature-Assisted One-Class Learning for Artist Authentication in Historical Drawings
• Quantum Autoencoder for Multivariate Time Series Anomaly Detection
• Fault Detection in New Wind Turbines with Limited Data by Generative Transfer Learning
• Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Think
• Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection
• DConAD: A Differencing-based Contrastive Representation Learning Framework for Time Series Anomaly Detection
• Unmanned Aerial Vehicle Flight Data Anomaly Detection and Recovery Prediction Based on Spatio-Temporal Correlation
• Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxels
• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detection
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems. [europepmc]
• On the nature and types of anomalies: a review of deviations in data. [europepmc]
• Multi-Perspective Anomaly Detection. [europepmc]
• Deep Convolutional Clustering-Based Time Series Anomaly Detection. [europepmc]
• Detecting failure modes in image reconstructions with interval neural network uncertainty. [europepmc]
• Robust, Primitive, and Unsupervised Quality Estimation for Segmentation Ensembles. [europepmc]
• Pediatric Otoscopy Video Screening With Shift Contrastive Anomaly Detection. [europepmc]
• DNA methylation-based classification of sinonasal tumors. [europepmc]
• Decentralized Real-Time Anomaly Detection in Cyber-Physical Production Systems under Industry Constraints. [europepmc]
• Canonical Response Parameterization: Quantifying the structure of responses to single-pulse intracranial electrical brain stimulation. [europepmc]
• Hazards&Robots: A dataset for visual anomaly detection in robotics. [europepmc]
• PPSW-SHAP: Towards Interpretable Cell Classification Using Tree-Based SHAP Image Decomposition and Restoration for High-Throughput Bright-Field Imaging. [europepmc]
• Deep learning-based prediction of in-hospital mortality for sepsis. [europepmc]
• Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles. [europepmc]
• Dense Out-of-Distribution Detection by Robust Learning on Synthetic Negative Data. [europepmc]
• Machine Learning Assisted Hit Prioritization for High Throughput Screening in Drug Discovery. [europepmc]
• Identifying Rare Events in Quantum Molecular Dynamics of Nanomaterials with Outlier Detection Indices. [europepmc]
• Anomaly detection via Gumbel Noise Score Matching. [europepmc]
• NiReject: toward automated bad channel detection in functional near-infrared spectroscopy. [europepmc]
• Research on a Novel Unsupervised-Learning-Based Pipeline Leak Detection Method Based on Temporal Kolmogorov-Arnold Network with Autoencoder Integration. [europepmc]
• Artificial intelligence for modeling and understanding extreme weather and climate events. [europepmc]
• Siamese network with change awareness for surface defect segmentation in complex backgrounds. [europepmc]
• Diffusion MRI with Machine Learning. [europepmc]
• EODA: A three-stage efficient outlier detection approach using Boruta-RF feature selection and enhanced KNN-based clustering algorithm. [europepmc]
• A swin transformer-based hybrid reconstruction discriminative network for image anomaly detection. [europepmc]

Related

• Deep Learning for Anomaly Detection: A Survey
• Deep Learning Advancements in Anomaly Detection: A Comprehensive Survey
• Deep Learning for Anomaly Detection: A Review
• Rethinking Assumptions in Deep Anomaly Detection
• Comparison of Anomaly Detectors: Context Matters