Anomaly Object Segmentation with Vision-Language Models for Steel Scrap Recycling

2025/06/16 by Tanaka, Daichi, Karasawa, Takumi, Takenouchi, Shu +1
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2506.13282

Abstract

Recycling steel scrap can reduce carbon dioxide (CO2) emissions from the steel industry. However, a significant challenge in steel scrap recycling is the inclusion of impurities other than steel. To address this issue, we propose vision-language-model-based anomaly detection where a model is finetuned in a supervised manner, enabling it to handle niche objects effectively. This model enables automated detection of anomalies at a fine-grained level within steel scrap. Specifically, we finetune the image encoder, equipped with multi-scale mechanism and text prompts aligned with both normal and anomaly images. The finetuning process trains these modules using a multiclass classification as the supervision.

Citations

• The Road Less Scheduled
• ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation
• AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model
• APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD
• Micrograph segmentations for DDEVD
• Segment Anything
• WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation
• Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• Towards Total Recall in Industrial Anomaly Detection
• Learning Transferable Visual Models From Natural Language Supervision
• PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and\n Localization
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Dice Loss for Data-imbalanced NLP Tasks
• Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder\n Anomaly Detection
• Class-Balanced Loss Based on Effective Number of Samples
• GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training
• Mask R-CNN
• Pyramid Scene Parsing Network

Related

• AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models
• One-class Steel Detector Using Patch GAN Discriminator for Visualising Anomalous Feature Map
• FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Model
• Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection
• Anomaly Detection by Adapting a pre-trained Vision Language Model