Video Anomaly Detection by Solving Decoupled Spatio-Temporal Jigsaw Puzzles

2022/07/20 by Guodong Wang, Yunhong Wang, Wang, Guodong +9 · 9 citations
Computer Science · #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Digital Media Forensic Detection #FOS: Computer and information sciences #Human Pose and Action Recognition

paper · pdf · doi:10.48550/arxiv.2207.10172

openalex publication_date 2022/07/20 · openalex created_date 2022/07/23 · openalex updated_date 2026/07/28

Abstract

Video Anomaly Detection (VAD) is an important topic in computer vision. Motivated by the recent advances in self-supervised learning, this paper addresses VAD by solving an intuitive yet challenging pretext task, i.e., spatio-temporal jigsaw puzzles, which is cast as a multi-label fine-grained classification problem. Our method exhibits several advantages over existing works: 1) the spatio-temporal jigsaw puzzles are decoupled in terms of spatial and temporal dimensions, responsible for capturing highly discriminative appearance and motion features, respectively; 2) full permutations are used to provide abundant jigsaw puzzles covering various difficulty levels, allowing the network to distinguish subtle spatio-temporal differences between normal and abnormal events; and 3) the pretext task is tackled in an end-to-end manner without relying on any pre-trained models. Our method outperforms state-of-the-art counterparts on three public benchmarks. Especially on ShanghaiTech Campus, the result is superior to reconstruction and prediction-based methods by a large margin.

Cited by

• Curriculum Multi-Task Self-Supervision Improves Lightweight Architectures for Onboard Satellite Hyperspectral Image Segmentation
• Action Hints: Semantic Typicality and Context Uniqueness for Generalizable Skeleton-based Video Anomaly Detection
• A Survey on Video Anomaly Detection via Deep Learning: Human, Vehicle, and Environment
• ALFred: An Active Learning Framework for Real-world Semi-supervised Anomaly Detection with Adaptive Thresholds
• The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM
• VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understanding
• Autoregressive Denoising Score Matching is a Good Video Anomaly Detector
• Sequential keypoint density estimator: an overlooked baseline of skeleton-based video anomaly detection
• Jigsaw-Puzzles: From Seeing to Understanding to Reasoning in Vision-Language Models

Related

• Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
• Self-Supervised Representation Learning for Visual Anomaly Detection
• Exploiting Spatial-temporal Correlations for Video Anomaly Detection
• Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection
• Real-world Video Anomaly Detection by Extracting Salient Features in Videos