Deep Transform and Metric Learning Network: Wedding Deep Dictionary Learning and Neural Networks

2020/02/18 by Wen Tang, Tang, Wen, Émilie Chouzenoux +5 · 2 citations
Computer Science · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Face and Expression Recognition #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2002.07898

openalex publication_date 2020/02/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

On account of its many successes in inference tasks and denoising applications, Dictionary Learning (DL) and its related sparse optimization problems have garnered a lot of research interest. While most solutions have focused on single layer dictionaries, the improved recently proposed Deep DL (DDL) methods have also fallen short on a number of issues. We propose herein, a novel DDL approach where each DL layer can be formulated as a combination of one linear layer and a Recurrent Neural Network (RNN). The RNN is shown to flexibly account for the layer-associated and learned metric. Our proposed work unveils new insights into Neural Networks and DDL and provides a new, efficient and competitive approach to jointly learn a deep transform and a metric for inference applications. Extensive experiments are carried out to demonstrate that the proposed method can not only outperform existing DDL but also state-of-the-art generic CNNs.

Citations

• Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems
• Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
• rm K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
• Wide Residual Networks
• Learning Multiple Layers of Features from Tiny Images

Cited by

• AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features
• Proximal Vision Transformer: Enhancing Feature Representation through Two-Stage Manifold Geometry

Related

• Deep Transform and Metric Learning Networks
• How to Train Your Deep Neural Network with Dictionary Learning
• Greedy Deep Dictionary Learning
• Majorization Minimization Technique for Optimally Solving Deep Dictionary Learning
• When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data