Describing Textures in the Wild
2013/11/14 by Mircea Cimpoi, Subhransu Maji, Cimpoi, Mircea +7 · 217 citations
Computer Science · #Generative Adversarial Networks and Image Synthesis #Multimodal Machine Learning Applications #Visual Attention and Saliency Detection
paper · pdf · doi:10.48550/arxiv.1311.3618
Abstract
Patterns and textures are defining characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly. Aiming at supporting this analytical dimension in image understanding, we address the challenging problem of describing textures with semantic attributes. We identify a rich vocabulary of forty-seven texture terms and use them to describe a large dataset of patterns collected in the wild.The resulting Describable Textures Dataset (DTD) is the basis to seek for the best texture representation for recognizing describable texture attributes in images. We port from object recognition to texture recognition the Improved Fisher Vector (IFV) and show that, surprisingly, it outperforms specialized texture descriptors not only on our problem, but also in established material recognition datasets. We also show that the describable attributes are excellent texture descriptors, transferring between datasets and tasks; in particular, combined with IFV, they significantly outperform the state-of-the-art by more than 8 percent on both FMD and KTHTIPS-2b benchmarks. We also demonstrate that they produce intuitive descriptions of materials and Internet images.
Cited by
- Rethinking Expert Training for Model Merging with Prompt Learning
- FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence
- Anomaly Detection by Effectively Leveraging Synthetic Images
- Decomposing Task Vectors for Refined Model Editing
- Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
- UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texture
- Model Merging via Multi-Teacher Knowledge Distillation
- Auxiliary Descriptive Knowledge for Few-Shot Adaptation of Vision-Language Model
- Retrieval-augmented Prompt Learning for Pre-trained Foundation Models
- SigLino: Efficient Multi-Teacher Distillation for Agglomerative Vision Foundation Models
- AmPLe: Supporting Vision-Language Models via Adaptive-Debiased Ensemble Multi-Prompt Learning
- TTP: Test-Time Padding for Adversarial Detection and Robust Adaptation on Vision-Language Models
- Open Ad-hoc Categorization with Contextualized Feature Learning
- SuperCLIP: CLIP with Simple Classification Supervision
- Calibrating Uncertainty for Zero-Shot Adversarial CLIP
- Predictive Sample Assignment for Semantically Coherent Out-of-Distribution Detection
- MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
- Advancing Cache-Based Few-Shot Classification via Patch-Driven Relational Gated Graph Attention
- Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective
- Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
- Decoupling Template Bias in CLIP: Harnessing Empty Prompts for Enhanced Few-Shot Learning
- PVeRA: Probabilistic Vector-Based Random Matrix Adaptation
- Dropout Prompt Learning: Towards Robust and Adaptive Vision-Language Models
- Greedy Alignment Principle for Optimizer Selection
- Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Models
- Fourier-Attentive Representation Learning: A Fourier-Guided Framework for Few-Shot Generalization in Vision-Language Models
- How (Mis)calibrated is Your Federated CLIP and What To Do About It?
- Domain Feature Collapse: Implications for Out-of-Distribution Detection and Solutions
- Basis-Oriented Low-rank Transfer for Few-Shot and Test-Time Adaptation
- SVRG and Beyond via Posterior Correction
- Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models
- Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging
- Hierarchical Semantic Alignment for Image Clustering
- Partially Shared Concept Bottleneck Models
- From Coefficients to Directions: Rethinking Model Merging with Directional Alignment
- PowerCLIP: Powerset Alignment for Contrastive Pre-Training
- From Points to Clouds: Learning Robust Semantic Distributions for Multi-modal Prompts
- Improving Robotic Manipulation Robustness via NICE Scene Surgery
- VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models
- AnoRefiner: Anomaly-Aware Group-Wise Refinement for Zero-Shot Industrial Anomaly Detection
- AnchorOPT: Towards Optimizing Dynamic Anchors for Adaptive Prompt Learning
- MetaRank: Task-Aware Metric Selection for Model Transferability Estimation
- RankOOD -- Class Ranking-based Out-of-Distribution Detection
- Annotation-Free Class-Incremental Learning
- CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selection
- FAST: Topology-Aware Frequency-Domain Distribution Matching for Coreset Selection
- Escaping Optimization Stagnation: Taking Steps Beyond Task Arithmetic via Difference Vectors
- ATAC: Augmentation-Based Test-Time Adversarial Correction for CLIP
- ReBaPL: Repulsive Bayesian Prompt Learning
- SafeR-CLIP: Mitigating NSFW Content in Vision-Language Models While Preserving Pre-Trained Knowledge
- TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models
- Exploiting Inter-Sample Information for Long-tailed Out-of-Distribution Detection
- TSRE: Channel-Aware Typical Set Refinement for Out-of-Distribution Detection
- BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapse
- HiGFA: Hierarchical Guidance for Fine-grained Data Augmentation with Diffusion Models
- Defending Unauthorized Model Merging via Dual-Stage Weight Protection
- Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification
- Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models
- Human-Corrected Labels Learning: Enhancing Labels Quality via Human Correction of VLMs Discrepancies
- Doubly Debiased Test-Time Prompt Tuning for Vision-Language Models
- Where and What Matters: Sensitivity-Aware Task Vectors for Many-Shot Multimodal In-Context Learning
- VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusion
- Noise & pattern: identity-anchored Tikhonov regularization for robust structural anomaly detection
- Towards Frequency-Adaptive Learning for SAR Despeckling
- SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking
- Decoupling Augmentation Bias in Prompt Learning for Vision-Language Models
- Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies
- GAFD-CC: Global-Aware Feature Decoupling with Confidence Calibration for OOD Detection
- SciTextures: Collecting and Connecting Visual Patterns, Models, and Code Across Science and Art
- Bayesian Natural Gradient Fine-Tuning of CLIP Models via Kalman Filtering
- Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection
- FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts
- LGCA: Enhancing Semantic Representation via Progressive Expansion
- Soft Task-Aware Routing of Experts for Equivariant Representation Learning
- A Retrospect to Multi-prompt Learning across Vision and Language
- A-TPT: Angular Diversity Calibration Properties for Test-Time Prompt Tuning of Vision-Language Models
- ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts
- Detecting CSAM Text-to-Image LoRAs From Weights
- MIN-Merging: Merge the Important Neurons for Model Merging
- Attentive multilayer fusion for vision transformers
- On the Provable Importance of Gradients for Language-Assisted Image Clustering
- UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation
- Enhancing CLIP Robustness via Cross-Modality Alignment
- VIPAMIN: Visual Prompt Initialization via Embedding Selection and Subspace Expansion
- Implicit Modeling for Transferability Estimation of Vision Foundation Models
- Head Pursuit: Probing Attention Specialization in Multimodal Transformers
- VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
- Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models
- Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Models
- Quantized FCA: Efficient Zero-Shot Texture Anomaly Detection
- Revisiting Logit Distributions for Reliable Out-of-Distribution Detection
- Class-Aware Prototype Learning with Negative Contrast for Test-Time Adaptation of Vision-Language Models
- Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
- Latent Space Factorization in LoRA
- VeFA: Vector-Based Feature Space Adaptation for Robust Model Fine-Tuning
- FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
- Ninja Codes: Neurally Generated Fiducial Markers for Stealthy 6-DoF Tracking
- Learning Task-Agnostic Representations through Multi-Teacher Distillation
- Elastic ViTs from Pretrained Models without Retraining
- GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection
- Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring
- Dissecting Mahalanobis: How Feature Geometry and Normalization Shape OOD Detection
- Exploring Cross-Modal Flows for Few-Shot Learning
- Weight Weaving: Parameter Pooling for Data-Free Model Merging
- Local Background Features Matter in Out-of-Distribution Detection
- Exploring and Leveraging Class Vectors for Classifier Editing
- microCLIP: Unsupervised CLIP Adaptation via Coarse-Fine Token Fusion for Fine-Grained Image Classification
- ΔEnergy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalization
- Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection
- Cooperative Pseudo Labeling for Unsupervised Federated Classification
- Neural Particle Automata: Learning Self-Organizing Particle Dynamics
- Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation
- PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learning
- D-TPT: Dimensional Entropy Maximization for Calibrating Test-Time Prompt Tuning in Vision-Language Models
- Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation
- TTRV: Test-Time Reinforcement Learning for Vision Language Models
- Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statistics
- How NOT to benchmark your SITE metric: Beyond Static Leaderboards and Towards Realistic Evaluation
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection
- Out-of-Distribution Detection from Small Training Sets using Bayesian Neural Network Classifiers
- Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learning
- Activation Quantization of Vision Encoders Needs Prefixing Registers
- Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models
- Intuitions of Machine Learning Researchers about Transfer Learning for Medical Image Classification
- Zero-Shot Decentralized Federated Learning
- TDHook: A Lightweight Framework for Interpretability
- GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learning
- Merge Now, Regret Later: The Hidden Cost of Model Merging is Adversarial Transferability
- Training-Free Synthetic Data Generation with Dual IP-Adapter Guidance
- DAC-LoRA: Dynamic Adversarial Curriculum for Efficient and Robust Few-Shot Adaptation
- FAST: Foreground-aware Diffusion with Accelerated Sampling Trajectory for Segmentation-oriented Anomaly Synthesis
- ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Research
- Unifying Adversarially Robust Model Experts in Vision-Language Models
- Transporting Task Vectors across Different Architectures without Training
- Alternating Training-based Label Smoothing Enhances Prompt Generalization
- No Labels Needed: Zero-Shot Image Classification with Collaborative Self-Learning
- TensLoRA: Tensor Alternatives for Low-Rank Adaptation
- Accurate and Efficient Low-Rank Model Merging in Core Space
- Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning
- Variational Task Vector Composition
- Long-Tailed Out-of-Distribution Detection with Refined Separate Class Learning
- Trade-offs in Cross-Domain Generalization of Foundation Model Fine-Tuned for Biometric Applications
- Which Direction to Choose? An Analysis on the Representation Power of Self-Supervised ViTs in Downstream Tasks
- Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models
- Self Identity Mapping
- An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
- Sharpness-Aware Geometric Defense for Robust Out-Of-Distribution Detection
- Double Helix Diffusion for Cross-Domain Anomaly Image Generation
- Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning
- A Modern Look at Simplicity Bias in Image Classification Tasks
- AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models
- Spectral Bottleneck in Sinusoidal Representation Networks: Noise is All You Need
- Tackling the Noisy Elephant in the Room: Label Noise-robust Out-of-Distribution Detection via Loss Correction and Low-rank Decomposition
- A brain-inspired paradigm for scalable quantum vision
- AttriPrompt: Dynamic Prompt Composition Learning for CLIP
- Hyperbolic Large Language Models
- DCV-ROOD Evaluation Framework: Dual Cross-Validation for Robust Out-of-Distribution Detection
- Towards Open World Detection: A Survey
- ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
- Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
- Causality-guided Prompt Learning for Vision-language Models via Visual Granulation
- Singular Value Few-shot Adaptation of Vision-Language Models
- FedAPT: Federated Adversarial Prompt Tuning for Vision-Language Models
- Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection
- Language-Aware Information Maximization for Transductive Few-Shot CLIP
- Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
- Activation Subspaces for Out-of-Distribution Detection
- Multi-Method Ensemble for Out-of-Distribution Detection
- Representation Learning with Adaptive Superpixel Coding
- Rethinking Layer-wise Model Merging through Chain of Merges
- Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
- UNIFORM: Unifying Knowledge from Large-scale and Diverse Pre-trained Models
- Backpropagation-Free Test-Time Adaptation via Probabilistic Gaussian Alignment
- Efficient Multi-Source Knowledge Transfer by Model Merging
- Stabilizing Open-Set Test-Time Adaptation via Primary-Auxiliary Filtering and Knowledge-Integrated Prediction
- Transfer learning optimization based on evolutionary selective fine tuning
- Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projector
- Preserve and Sculpt: Manifold-Aligned Fine-tuning of Vision-Language Models for Few-Shot Learning
- Cross-Domain Few-Shot Learning via Multi-View Collaborative Optimization with Vision-Language Models
- AdaRing: Towards Ultra-Light Vision-Language Adaptation via Cross-Layer Tensor Ring Decomposition
- Noise Matters: Optimizing Matching Noise for Diffusion Classifiers
- SemPT: Semantic Prompt Tuning for Vision-Language Models
- Fourier-Guided Attention Upsampling for Image Super-Resolution
- Retrieval-Augmented Prompt for OOD Detection
- Out-of-Distribution Detection using Counterfactual Distance
- AME: Aligned Manifold Entropy for Robust Vision-Language Distillation
- Transferable Model-agnostic Vision-Language Model Adaptation for Efficient Weak-to-Strong Generalization
- Adaptive Cache Enhancement for Test-Time Adaptation of Vision-Language Models
- Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classification
- Separation and Collaboration: Two-Level Routing Grouped Mixture-of-Experts for Multi-Domain Continual Learning
- BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Models
- Enhancing Small-Scale Dataset Expansion with Triplet-Connection-based Sample Re-Weighting
- Adapting Vision-Language Models Without Labels: A Comprehensive Survey
- Accelerating Conditional Prompt Learning via Masked Image Modeling for Vision-Language Models
- ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updates
- Generalized Few-Shot Out-of-Distribution Detection
- Robust Prompt Tuning for Vision-Language Models with Mild Semantic Noise
- AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization
- UniFGVC: Universal Training-Free Few-Shot Fine-Grained Vision Classification via Attribute-Aware Multimodal Retrieval
- RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging
- Pseudo-label Induced Subspace Representation Learning for Robust Out-of-Distribution Detection
- Causal Disentanglement and Cross-Modal Alignment for Enhanced Few-Shot Learning
- Uncertainty-Guided Face Matting for Occlusion-Aware Face Transformation
- REFLECT: Rectified Flows for Efficient Brain Anomaly Correction Transport
- Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models
- EvoVLMA: Evolutionary Vision-Language Model Adaptation
- RouteMark: A Fingerprint for Intellectual Property Attribution in Routing-based Model Merging
- Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Models
- DisTaC: Conditioning Task Vectors via Distillation for Robust Model Merging
- COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition
- Decouple before Align: Visual Disentanglement Enhances Prompt Tuning
- Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product
- BOOD: Boundary-based Out-Of-Distribution Data Generation
- Zero-Shot Image Anomaly Detection Using Generative Foundation Models
- COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
- MSGCoOp: Multiple Semantic-Guided Context Optimization for Few-Shot Learning
- Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration
Related