Using predefined vector systems as latent space configuration for neural network supervised training on data with arbitrarily large number of classes

2025/10/05 by Nikita Gabdullin, Gabdullin, Nikita · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face recognition and analysis #Machine Learning (cs.LG) #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.2510.04090

openalex publication_date 2025/10/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Supervised learning (SL) methods are indispensable for neural network (NN) training used to perform classification tasks. While resulting in very high accuracy, SL training often requires making NN parameter number dependent on the number of classes, limiting their applicability when the number of classes is extremely large or unknown in advance. In this paper we propose a methodology that allows one to train the same NN architecture regardless of the number of classes. This is achieved by using predefined vector systems as the target latent space configuration (LSC) during NN training. We discuss the desired properties of target configurations and choose randomly perturbed vectors of An root system for our experiments. These vectors are used to successfully train encoders and visual transformers (ViT) on Cinic-10 and ImageNet-1K in low- and high-dimensional cases by matching NN predictions with the predefined vectors. Finally, ViT is trained on a dataset with 1.28 million classes illustrating the applicability of the method to training on datasets with extremely large number of classes. In addition, potential applications of LSC in lifelong learning and NN distillation are discussed illustrating versatility of the proposed methodology.

Citations

• The effects of Hessian eigenvalue spectral density type on the applicability of Hessian analysis to generalization capability assessment of neural networks
• GPT-4o System Card
• Latent space configuration for improved generalization in supervised autoencoder neural networks
• A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge
• A Comprehensive Survey of Continual Learning: Theory, Method and Application
• Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
• CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels
• Emerging Properties in Self-Supervised Vision Transformers
• Emerging Properties in Self-Supervised Vision Transformers
• DER: Dynamically Expandable Representation for Class Incremental Learning
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Bootstrap your own latent: A new approach to self-supervised Learning
• Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
• Modeling Teacher-Student Techniques in Deep Neural Networks for Knowledge Distillation
• Momentum Contrast for Unsupervised Visual Representation Learning
• Momentum Contrast for Unsupervised Visual Representation Learning
• RandAugment: Practical automated data augmentation with a reduced search\n space
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Infinite Mixture Prototypes for Few-Shot Learning
• Deep Clustering for Unsupervised Learning of Visual Features
• Learning towards Minimum Hyperspherical Energy
• CosFace: Large Margin Cosine Loss for Deep Face Recognition
• Sampling Matters in Deep Embedding Learning
• SphereFace: Deep Hypersphere Embedding for Face Recognition
• Prototypical Networks for Few-shot Learning
• iCaRL: Incremental Classifier and Representation Learning
• Person Re-identification: Past, Present and Future
• MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
• Progressive Neural Networks
• Matching Networks for One Shot Learning
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Distilling the Knowledge in a Neural Network
• ImageNet classification with deep convolutional neural networks

Cited by

• Exploring possible vector systems for faster training of neural networks with preconfigured latent spaces
• Series of quasi-uniform scatterings with fast search, root systems and neural network classifications

Related

• Exploring possible vector systems for faster training of neural networks with preconfigured latent spaces
• Mixing Real and Synthetic Data to Enhance Neural Network Training -- A Review of Current Approaches
• SWAT-NN: Simultaneous Weights and Architecture Training for Neural Networks in a Latent Space
• EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones
• No Reason for No Supervision: Improved Generalization in Supervised Models