rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
2006/10/18 by M. Aharon, Michal Aharon, Michael Elad +3 · 316 citations
Computer Science · Engineering · #Blind Source Separation Techniques #Sparse and Compressive Sensing Techniques #Ultrasonics and Acoustic Wave Propagation
paper · doi:10.1109/tsp.2006.881199
openalex publication_date 2006/10/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
Abstract
In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signal-atoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, regularization in inverse problems, feature extraction, and more. Recent activity in this field has concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. Designing dictionaries to better fit the above model can be done by either selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Both of these techniques have been considered, but this topic is largely still open. In this paper we propose a novel algorithm for adapting dictionaries in order to achieve sparse signal representations. Given a set of training signals, we seek the dictionary that leads to the best representation for each member in this set, under strict sparsity constraints. We present a new method-the K-SVD algorithm-generalizing the K-means clustering process. K-SVD is an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data. The update of the dictionary columns is combined with an update of the sparse representations, thereby accelerating convergence. The K-SVD algorithm is flexible and can work with any pursuit method (e.g., basis pursuit, FOCUSS, or matching pursuit). We analyze this algorithm and demonstrate its results both on synthetic tests and in applications on real image data
Citations
Cited by
- Weighed l1 on the simplex: Compressive sensing meets locality
- Automatic scoring of apnea and hypopnea events using blood oxygen saturation signals
- Variations on the CSC model
- Dictionary learning for integrative, multimodal and scalable single-cell analysis
- NETT: Solving Inverse Problems with Deep Neural Networks
- A deep network for sinogram and CT image reconstruction
- Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation
- Model-based Sparse Coding beyond Gaussian Independent Model
- Ensemble Sparse Models for Image Analysis
- Adversarial Déjà Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks
- Provably Convergent Algorithms for Solving Inverse Problems Using Generative Models
- Learning to compress and search visual data in large-scale systems
- Quadratic Autoencoder (Q-AE) for Low-dose CT Denoising
- Discriminative models for robust image classification
- Order-preserving factor analysis (OPFA)
- Sparse Signal Recovery using Generalized Approximate Message Passing with Built-in Parameter Estimation
- Non-Negative Kernel Sparse Coding for the Classification of Motion Data
- Greedy Deep Dictionary Learning
- An unsupervised deep learning framework for medical image denoising
- Multi-Stage Progressive Image Restoration
- Correlation and Class Based Block Formation for Improved Structured Dictionary Learning
- Community-Level Anomaly Detection for Anti-Money Laundering
- Contour Sparse Representation with SDD Features for Object Recognition
- Low-Dose CT Image Denoising Using Parallel-Clone Networks
- COSPADI: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning
- A Framework for the Analysis of Computational Imaging Systems with Practical Applications
- Cascaded Channel Estimation for Large Intelligent Metasurface Assisted Massive MIMO
- Active Dictionary Learning in Sparse Representation Based Classification
- One-Class Classification: A Survey
- Reprogramming pretrained language models for protein sequence representation learning
- Sequential adaptive compressed sampling via Huffman codes
- Efficient Blind Compressed Sensing Using Sparsifying Transforms with Convergence Guarantees and Application to MRI
- Transfer Learning from Synthetic to Real-Noise Denoising with Adaptive Instance Normalization
- Adaptive noise imitation for image denoising
- Dynamic Upsampling of Smoke through Dictionary-based Learning
- Fast and Effective Algorithms for Symmetric Nonnegative Matrix Factorization
- Numerical methods for phase retrieval
- Analysis of Inpainting via Clustered Sparsity and Microlocal Analysis
- Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches
- Online and Stable Learning of Analysis Operators
- Concave losses for robust dictionary learning
- Analyzing Linear Dynamical Systems: From Modeling to Coding and Learning
- Progressive Identification of True Labels for Partial-Label Learning
- Self-Expressive Decompositions for Matrix Approximation and Clustering
- Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition
- Generic Image Classification Approaches Excel on Face Recognition
- Convex relaxations of structured matrix factorizations
- Learned convex regularizers for inverse problems
- Structured Dictionary Learning for Classification
- Alternating minimization for dictionary learning: Local Convergence Guarantees
- Image Segmentation Using Subspace Representation and Sparse Decomposition
- A comprehensive study of sparse representation techniques for offline signature verification
- Convolutional versus Self-Organized Operational Neural Networks for Real-World Blind Image Denoising
- Sparse Representation based Multi-sensor Image Fusion: A Review
- Sparse Autoencoder Neural Operators: Model Recovery in Function Spaces
- Deep Learning for Low-Dose CT Denoising
- Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
- A Block Decomposition Algorithm for Sparse Optimization
- Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning
- NBNet: Noise Basis Learning for Image Denoising with Subspace Projection
- CycleISP: Real Image Restoration via Improved Data Synthesis
- Spatial-Adaptive Network for Single Image Denoising
- Sparse Signal Subspace Decomposition Based on Adaptive Over-complete Dictionary
- Deep Learning on Image Denoising: An overview
- Covariance-domain Dictionary Learning for Overcomplete EEG Source Identification
- P-SIF: Document Embeddings Using Partition Averaging
- Sparse image representation by discrete cosine/spline based dictionaries
- A Local Block Coordinate Descent Algorithm for the Convolutional Sparse Coding Model
- JOBS: Joint-Sparse Optimization from Bootstrap Samples
- Learning ℓ1-based analysis and synthesis sparsity priors using bi-level optimization
- SemiContour: A Semi-supervised Learning Approach for Contour Detection
- Learning Transformations for Classification Forests
- Deep Transform and Metric Learning Network: Wedding Deep Dictionary Learning and Neural Networks
- Non-negative representation based discriminative dictionary learning for face recognition
- Multi-task additive models with shared transfer functions based on dictionary learning
- Anomaly-Sensitive Dictionary Learning for Unsupervised Diagnostics of Solid Media
- Dictionary Learning Using Rank-One Projection (ROP)
- Group-based Sparse Representation for Image Compressive Sensing Reconstruction with Non-Convex Regularization
- On-Demand Learning for Deep Image Restoration
- Sparse and redundant signal representations for x-ray computed tomography
- Lensless Compressive Imaging
- Learning to Optimize: A Primer and A Benchmark
- Gromov-Wasserstein Factorization Models for Graph Clustering
- Interactive Character Posing by Sparse Coding
- Networked Estimation using Sparsifying Basis Prediction
- Double-Sparsity Learning Based Channel-and-Signal Estimation in Massive MIMO with Generalized Spatial Modulation
- Multiscale Dictionary Learning: Non-Asymptotic Bounds and Robustness
- Learning Raw Image Denoising with Bayer Pattern Unification and Bayer Preserving Augmentation
- Sparse Dictionary-based Attributes for Action Recognition and Summarization
- A Dual Sparse Decomposition Method for Image Denoising
- Localization, Decomposition, and Dictionary Learning of Piecewise-Constant Signals on Graphs
- Stabilizing Deep Tomographic Reconstruction
- Optimized Kernel-based Projection Space of Riemannian Manifolds
- Online Discriminative Dictionary Learning for Image Classification Based on Block-Coordinate Descent Method
- Boosting Dictionary Learning with Error Codes
- Fully Trainable and Interpretable Non-Local Sparse Models for Image Restoration
- Performance Limits of Dictionary Learning for Sparse Coding
- Sparse Random Approximation and Lossy Compression
- Posterior Sampling for Image Restoration using Explicit Patch Priors
- Sparse Illumination Learning and Transfer for Single-Sample Face Recognition with Image Corruption and Misalignment
- Spatiotemporal KSVD Dictionary Learning for Online Multi-target Tracking
- Continuous Operator Authentication for Teleoperated Systems Using Hidden Markov Models
- The Learning and Prediction of Application-level Traffic Data in Cellular Networks
- Exact Recovery of Sparsely-Used Dictionaries
- Hybrid Function Sparse Representation towards Image Super Resolution
- A dictionary learning and source recovery based approach to classify diverse audio sources
- Domain Adaptation through Synthesis for Unsupervised Person Re-identification
- Sparseness helps: Sparsity Augmented Collaborative Representation for Classification
- Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
- Global Optimality in Tensor Factorization, Deep Learning, and Beyond
- Robust Multi-subspace Analysis Using Novel Column L0-norm Constrained Matrix Factorization
- Analysis Dictionary Learning: An Efficient and Discriminative Solution
- Urban spatial-temporal activity structures: a New Approach to Inferring the Intra-urban Functional Regions via Social Media Check-In Data
- Online Dictionary Learning Aided Target Recognition In Cognitive GPR
- Non-Convex Weighted Lp Nuclear Norm based ADMM Framework for Image Restoration
- Approximate Guarantees for Dictionary Learning
- Blind calibration for compressed sensing by convex optimization
- On the convergence of group-sparse autoencoders
- 1-Bit Compressive Sensing via Approximate Message Passing with Built-in Parameter Estimation
- Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
- Learned Multi-layer Residual Sparsifying Transform Model for Low-dose CT Reconstruction
- Information-theoretic Dictionary Learning for Image Classification
- Sparse Coding Approach for Multi-Frame Image Super Resolution
- DCFNet: Deep Neural Network with Decomposed Convolutional Filters
- ForestHash: Semantic Hashing With Shallow Random Forests and Tiny Convolutional Networks
- Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery
- Model Function Based Conditional Gradient Method with Armijo-like Line Search
- Low-light Image Restoration with Short- and Long-exposure Raw Pairs
- Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline
- Learning Discriminative Multilevel Structured Dictionaries for Supervised Image Classification
- High Speed and High Dynamic Range Video with an Event Camera
- Denoising and Completion of 3D Data via Multidimensional Dictionary Learning
- Synthesis-based Robust Low Resolution Face Recognition
- Poisson noise reduction with non-local PCA
- A survey of dimensionality reduction techniques
- Can fully convolutional networks perform well for general image restoration problems?
- K-Deep Simplex: Deep Manifold Learning via Local Dictionaries
- Boosting of Image Denoising Algorithms
- Inducing Hierarchical Compositional Model by Sparsifying Generator Network
- EnTrans:Leveraging Kinetic Energy Harvesting Signal for Transportation Mode Detection
- P3: Toward Privacy-Preserving Photo Sharing
- Reprogramming Language Models for Molecular Representation Learning
- An end-to-end convolutional selective autoencoder approach to Soybean Cyst Nematode eggs detection
- Local Identification of Overcomplete Dictionaries
- Unsupervised Image Denoising with Frequency Domain Knowledge
- Image Super-Resolution Using Deep Convolutional Networks
- Semi-supervised Sparse Representation with Graph Regularization for Image Classification
- Discriminative Localized Sparse Representations for Breast Cancer Screening
- Progressive Learning for Systematic Design of Large Neural Networks
- A Non-Local Means Approach for Gaussian Noise Removal from Images using a Modified Weighting Kernel
- Online unsupervised deep unfolding for MIMO channel estimation
- A sparse-sampling approach for the fast computation of matrices: application to molecular vibrations
- Side Information for Face Completion: a Robust PCA Approach
- When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach
- Kernel Recursive Least Squares Dictionary Learning Algorithm
- Deep Micro-Dictionary Learning and Coding Network
- FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
- Discriminative Local Sparse Representation by Robust Adaptive Dictionary Pair Learning
- Locality Constrained Analysis Dictionary Learning via K-SVD Algorithm
- Phase-Optimized K-SVD for Signal Extraction from Underdetermined Multichannel Sparse Mixtures
- Discrete Multi-modal Hashing with Canonical Views for Robust Mobile Landmark Search
- Inertial Block Proximal Methods for Non-Convex Non-Smooth Optimization
- Unsupervised Feature Learning for Dense Correspondences across Scenes
- Observable dictionary learning for high-dimensional statistical inference
- On the computation of tensor functions under tensor-tensor multiplications with linear maps
- Recovery and Generalization in Over-Realized Dictionary Learning
- Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion
- Dictionary Learning Using Rank-One Atomic Decomposition (ROAD)
- The Power of Complementary Regularizers: Image Recovery via Transform Learning and Low-Rank Modeling
- Earthquake Detection in 1-D Time Series Data with Feature Selection and Dictionary Learning
- A word recurrence based algorithm to extract genomic dictionaries
- Dictionary Learning with Almost Sure Error Constraints
- Learning a collaborative multiscale dictionary based on robust empirical mode decomposition
- Jitter-Adaptive Dictionary Learning - Application to Multi-Trial Neuroelectric Signals
- Multiscale Adaptive Representation of Signals: I. The Basic Framework
- Hyperspectral recovery from RGB images using Gaussian Processes
- Fisher Discriminative Least Squares Regression for Image Classification
- Unmixing Incoherent Structures of Big Data by Randomized or Greedy Decomposition
- D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
- Multimodal Sparse Bayesian Dictionary Learning
- Stochastic Greedy Algorithms For Multiple Measurement Vectors
- Learning to Invert: Signal Recovery via Deep Convolutional Networks
- Designing Incoherent Dictionaries for Compressed Sensing: Algorithm Comparison
- A convex variational model for learning convolutional image atoms from incomplete data
- Image Restoration and Reconstruction using Variable Splitting and Class-adapted Image Priors
- Removing rain streaks by a linear model
- Separable Dictionary Learning
- 4D Seismic History Matching Incorporating Unsupervised Learning
- Image Super-Resolution Using Deep Convolutional Networks
- Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees
- DECT-MULTRA: Dual-Energy CT Image Decomposition With Learned Mixed Material Models and Efficient Clustering
- Optimal Non-Linear Models for Sparsity and Sampling
- An eigenanalysis of data centering in machine learning
- Bayesian Image Restoration for Poisson Corrupted Image using a Latent Variational Method with Gaussian MRF
- Role-Aware Modeling for N-ary Relational Knowledge Bases
- Efficient Algorithms for Convolutional Inverse Problems in Multidimensional Imaging
- Image denoising via K-SVD with primal-dual active set algorithm
- Scalable Convolutional Dictionary Learning with Constrained Recurrent Sparse Auto-encoders
- Sparse Representation of 3D Images for Piecewise Dimensionality Reduction with High Quality Reconstruction
- Dictionary Learning with Low-rank Coding Coefficients for Tensor Completion
- Principal Basis Analysis in Sparse Representation
- A Matrix Splitting Method for Composite Function Minimization
- Simultaneous Fidelity and Regularization Learning for Image Restoration
- A Region Based Easy Path Wavelet Transform For Sparse Image Representation
- Sparse Coding on Cascaded Residuals
- Simple, Efficient, and Neural Algorithms for Sparse Coding
- A Novel Compressive Sensing based Enhanced Multiplexing Scheme for MIMO System
- Structural Group Sparse Representation for Image Compressive Sensing Recovery
- A Critical Analysis of Patch Similarity Based Image Denoising Algorithms
- Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN
- Analysis Co-Sparse Coding for Energy Disaggregation
- Clustering with Jointly Learned Nonlinear Transforms Over Discriminating Min-Max Similarity/Dissimilarity Assignment
- On the Local Correctness of L1 Minimization for Dictionary Learning
- Learning computationally efficient dictionaries and their implementation as fast transforms
- Reconstructing the Noise Manifold for Image Denoising
- Image representation and denoising using squared eigenfunctions of Schrodinger operator
- Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling
- DB-KSVD: Scalable Alternating Optimization for Disentangling High-Dimensional Embedding Spaces
- Practical Deep Raw Image Denoising on Mobile Devices
- Inference-Time Decomposition of Activations (ITDA): A Scalable Approach to Interpreting Large Language Models
- Scene-Adapted Plug-and-Play Algorithm with Guaranteed Convergence: Applications to Data Fusion in Imaging
- Optimizing the Latent Space of Generative Networks
- Low-Dose CT via Deep Neural Network
- Invertible Denoising Network: A Light Solution for Real Noise Removal
- RankMap: A Platform-Aware Framework for Distributed Learning from Dense Datasets
- Learning Structured Twin-Incoherent Twin-Projective Latent Dictionary Pairs for Classification
- Partitioned Shape Modeling with On-the-Fly Sparse Appearance Learning for Anterior Visual Pathway Segmentation
- Deep Representation with ReLU Neural Networks
- Structured Occlusion Coding for Robust Face Recognition
- On Collaborative Compressive Sensing Systems: The Framework, Design and Algorithm
- Data-driven audio recognition: a supervised dictionary approach
- More Algorithms for Provable Dictionary Learning
- Simplicity is Key: An Unsupervised Pretraining Approach for Sparse Radio Channels
- Parameterizing Region Covariance: An Efficient Way To Apply Sparse Codes On Second Order Statistics
- Sparse Time-Frequency decomposition by dictionary learning
- Full Reference Screen Content Image Quality Assessment by Fusing Multi-level Structure Similarity
- Signal Recovery from Pooling Representations
- Learning spatially-correlated temporal dictionaries for calcium imaging
- Dictionary Identification - Sparse Matrix-Factorisation via ℓ1-Minimisation
- Statistical Physics and Information Theory Perspectives on Linear Inverse Problems
- On Convex Duality in Linear Inverse Problems
- VIDOSAT: High-dimensional Sparsifying Transform Learning for Online Video Denoising
- Online Convolutional Sparse Coding with Sample-Dependent Dictionary
- Decentralized Dictionary Learning Over Time-Varying Digraphs
- An Attention-Based Approach for Single Image Super Resolution
- Fast Rotational Sparse Coding
- Online Convolutional Dictionary Learning for Multimodal Imaging
- Analyzing the Weighted Nuclear Norm Minimization and Nuclear Norm Minimization based on Group Sparse Representation
- SEGAN: Structure-Enhanced Generative Adversarial Network for Compressed Sensing MRI Reconstruction
- Compressed Sensing with Probability-based Prior Information
- Supervised Dictionary Learning
- A Low-rank Tensor Dictionary Learning Method for Multi-spectral Images Denoising
- Probabilistic semi-nonnegative matrix factorization: a Skellam-based framework
- Region-specific Dictionary Learning-based Low-dose Thoracic CT Reconstruction
- Multi-Resolution Compressed Sensing via Approximate Message Passing
- Recent Progress in Image Deblurring
- Label Embedded Dictionary Learning for Image Classification
- Intelligent Reflecting Surface for Massive Device Connectivity: Joint Activity Detection and Channel Estimation
- Learning Deep Convolutional Features for MRI Based Alzheimer's Disease Classification
- Two-layer clustering-based sparsifying transform learning for low-dose CT reconstruction
- Unsupervised Cross-Domain Recognition by Identifying Compact Joint Subspaces
- Supervised Dictionary Learning and Sparse Representation-A Review
- Compressive Estimation of Doubly Selective Channels in Multicarrier Systems: Leakage Effects and Sparsity-Enhancing Processing
- PADDLE: Proximal Algorithm for Dual Dictionaries LEarning
- Learning A Task-Specific Deep Architecture For Clustering
- A Bayesian Nonparametric Approach to Image Super-resolution
- Active Deep Learning for Classification of Hyperspectral Images
- Image Denoising by Random Interpolation Average with Low-Rank Matrix Approximation
- Interpretable Multiple-Kernel Prototype Learning for Discriminative Representation and Feature Selection
- A Probabilistic Framework for Discriminative Dictionary Learning
- Analysis of the Optimization Landscapes for Overcomplete Representation Learning
- On the Minimax Risk of Dictionary Learning
- Text Extraction From Texture Images Using Masked Signal Decomposition
- A latent factor model with a mixture of sparse and dense factors to model gene expression data with confounding effects
- Learning Dictionaries With Bounded Self-Coherence
- Dictionary-based Monitoring of Premature Ventricular Contractions: An Ultra-Low-Cost Point-of-Care Service
- Optimized Structured Sparse Sensing Matrices for Compressive Sensing
- Generative Modeling of Discrete Latent Structures via Dynamic Policy Gradients
- On the Computational Intractability of Exact and Approximate Dictionary Learning
- Dictionary Learning with Convex Update (ROMD)
- Online Learning with Many Experts
- Tree-Structure Bayesian Compressive Sensing for Video
- Learning Fast Dictionaries for Sparse Representations Using Low-Rank Tensor Decompositions
- See all by looking at a few: Sparse modeling for finding representative objects
- Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity
- Subspace metrics for multivariate dictionaries and application to EEG
- Up in the Air: When Homes Meet the Web of Things
- SRoll3: A neural network approach to reduce large-scale systematic effects in the Planck High-Frequency Instrument maps
- Greedy Signal Space Recovery Algorithm with Overcomplete Dictionaries in Compressive Sensing
- Complete Dictionary Learning via ℓp-norm Maximization
- A Method for Robust Online Classification using Dictionary Learning:\n Development and Assessment for Monitoring Manual Material Handling Activities\n Using Wearable Sensors
- Deep Identity-aware Transfer of Facial Attributes
- DeepLSR: a deep learning approach for laser speckle reduction
- Spatial-Aware Dictionary Learning for Hyperspectral Image Classification
- Weakly Supervised PatchNets: Describing and Aggregating Local Patches for Scene Recognition
- Application of dictionary learning to denoise LIGO’s blip noise transients
- Linearized Kernel Dictionary Learning
- Collaborative Filtering-Based Method for Low-Resolution and Details\n Preserving Image Denoising
- Audio Inpainting: Revisited and Reweighted
- Derivatives and inverse of cascaded linear+nonlinear neural models
- Image Restoration by Combined Order Regularization with Optimal Spatial Adaptation
- Fast dictionary learning from incomplete data
- Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products
- Face Aging Effect Simulation Using Hidden Factor Analysis Joint Sparse Representation
- A linear approach for sparse coding by a two-layer neural network
- Deep learning methods for solving linear inverse problems: Research directions and paradigms
- Subspace Estimation and Decomposition for Large Millimeter-Wave MIMO Systems
- Laplacian Change Point Detection for Dynamic Graphs
- Discriminative Nonlinear Analysis Operator Learning: When Cosparse Model Meets Image Classification
- Pseudo-Zernike Moments Based Sparse Representations for SAR Image Classification
- An improved analysis of the ER-SpUD dictionary learning algorithm
- DeepISP: Toward Learning an End-to-End Image Processing Pipeline
- Blind Universal Bayesian Image Denoising With Gaussian Noise Level Learning
- Group Sparsity Residual Constraint for Image Denoising
- Matrix Completion via Non-Convex Relaxation and Adaptive Correlation Learning
- Scalable Block-Diagonal Locality-Constrained Projective Dictionary Learning
- Insights Into Analysis Operator Learning: From Patch-Based Sparse Models to Higher Order MRFs
Related