Discriminative Learning of Similarity and Group Equivariant\n Representations

2018/08/29 by Shubhendu Trivedi, Trivedi, Shubhendu
Computer Science · #AI in cancer detection #FOS: Computer and information sciences #Face and Expression Recognition #Image Retrieval and Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1808.10078

openalex publication_date 2018/08/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

One of the most fundamental problems in machine learning is to compare\nexamples: Given a pair of objects we want to return a value which indicates\ndegree of (dis)similarity. Similarity is often task specific, and pre-defined\ndistances can perform poorly, leading to work in metric learning. However,\nbeing able to learn a similarity-sensitive distance function also presupposes\naccess to a rich, discriminative representation for the objects at hand. In\nthis dissertation we present contributions towards both ends. In the first part\nof the thesis, assuming good representations for the data, we present a\nformulation for metric learning that makes a more direct attempt to optimize\nfor the k-NN accuracy as compared to prior work. We also present extensions of\nthis formulation to metric learning for kNN regression, asymmetric similarity\nlearning and discriminative learning of Hamming distance. In the second part,\nwe consider a situation where we are on a limited computational budget i.e.\noptimizing over a space of possible metrics would be infeasible, but access to\na label aware distance metric is still desirable. We present a simple, and\ncomputationally inexpensive approach for estimating a well motivated metric\nthat relies only on gradient estimates, discussing theoretical and experimental\nresults. In the final part, we address representational issues, considering\ngroup equivariant convolutional neural networks (GCNNs). Equivariance to\nsymmetry transformations is explicitly encoded in GCNNs; a classical CNN being\nthe simplest example. In particular, we present a SO(3)-equivariant neural\nnetwork architecture for spherical data, that operates entirely in Fourier\nspace, while also providing a formalism for the design of fully Fourier neural\nnetworks that are equivariant to the action of any continuous compact group.\n

Citations

• Deep Convolutional Networks on Graph-Structured Data
• A tutorial on support vector regression
• Optimal Smoothing in Single-Index Models
• Consistent Nonparametric Regression
• Deep Learning in Neural Networks: An Overview
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Gradient-based learning applied to document recognition
• Backpropagation Applied to Handwritten Zip Code Recognition
• Distinctive Image Features from Scale-Invariant Keypoints
• Computing Fourier Transforms and Convolutions on the 2-Sphere
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• ShapeNet: An Information-Rich 3D Model Repository
• Learning in Implicit Generative Models
• Covariant Compositional Networks For Learning Graphs
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds

Related

• Learning Symmetrization for Equivariance with Orbit Distance Minimization
• Geometric Deep Learning and Equivariant Neural Networks
• Metric Learning for Clifford Group Equivariant Neural Networks
• Theoretical Aspects of Group Equivariant Neural Networks
• Equivariance with Learned Canonicalization Functions