Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation

2022/11/26 by Yuyuan Liu, Choubo Ding, Liu, Yuyuan +11 · 15 citations
Computer Science · Engineering · Medicine · #Computer Vision and Pattern Recognition (cs.CV) #Data-Driven Disease Surveillance #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Remote-Sensing Image Classification

paper · pdf · doi:10.48550/arxiv.2211.14512

openalex publication_date 2022/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Semantic segmentation models classify pixels into a set of known (``in-distribution'') visual classes. When deployed in an open world, the reliability of these models depends on their ability not only to classify in-distribution pixels but also to detect out-of-distribution (OoD) pixels. Historically, the poor OoD detection performance of these models has motivated the design of methods based on model re-training using synthetic training images that include OoD visual objects. Although successful, these re-trained methods have two issues: 1) their in-distribution segmentation accuracy may drop during re-training, and 2) their OoD detection accuracy does not generalise well to new contexts (e.g., country surroundings) outside the training set (e.g., city surroundings). In this paper, we mitigate these issues with: (i) a new residual pattern learning (RPL) module that assists the segmentation model to detect OoD pixels without affecting the inlier segmentation performance; and (ii) a novel context-robust contrastive learning (CoroCL) that enforces RPL to robustly detect OoD pixels among various contexts. Our approach improves by around 10% FPR and 7% AuPRC the previous state-of-the-art in Fishyscapes, Segment-Me-If-You-Can, and RoadAnomaly datasets. Our code is available at: https://github.com/yyliu01/RPL.

Cited by

• ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Data
• SupLID: Geometrical Guidance for Out-of-Distribution Detection in Semantic Segmentation
• Relative Energy Learning for LiDAR Out-of-Distribution Detection
• Leveraging Text-Driven Semantic Variation for Robust OOD Segmentation
• Training-Free Out-Of-Distribution Segmentation With Foundation Models
• From Pixel to Mask: A Survey of Out-of-Distribution Segmentation
• Uncertainty-Aware Likelihood Ratio Estimation for Pixel-Wise Out-of-Distribution Detection
• Distributional Uncertainty for Out-of-Distribution Detection
• Stereo-based 3D Anomaly Object Detection for Autonomous Driving: A New Dataset and Baseline
• Objectomaly: Objectness-Aware Refinement for OoD Segmentation with Structural Consistency and Boundary Precision
• CoT-Segmenter: Enhancing OOD Detection in Dense Road Scenes via Chain-of-Thought Reasoning
• OoDDINO:A Multi-level Framework for Anomaly Segmentation on Complex Road Scenes
• Out-of-Distribution Semantic Occupancy Prediction
• AuralSAM2: Enabling SAM2 Hear Through Pyramid Audio-Visual Feature Prompting
• Anomalies by Synthesis: Anomaly Detection using Generative Diffusion Models for Off-Road Navigation

Related

• Discriminative out-of-distribution detection for semantic segmentation
• Weakly Supervised Semantic Segmentation using Out-of-Distribution Data
• Fool Me Once: Robust Selective Segmentation via Out-of-Distribution\n Detection with Contrastive Learning
• Efficacy of Pixel-Level OOD Detection for Semantic Segmentation
• Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond