Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning

2025/10/16 by Shikuang Deng, Jiayuan Zhang, Deng, Shikuang +7
Computer Science · Engineering · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Neural Networks and Reservoir Computing

paper · pdf · doi:10.48550/arxiv.2510.14810

openalex publication_date 2025/10/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

Hebbian learning is a biological principle that intuitively describes how neurons adapt their connections through repeated stimuli. However, when applied to machine learning, it suffers serious issues due to the unconstrained updates of the connections and the lack of accounting for feedback mediation. Such shortcomings limit its effective scaling to complex network architectures and tasks. To this end, here we introduce the Structural Projection Hebbian Representation (SPHeRe), a novel unsupervised learning method that integrates orthogonality and structural information preservation through a local auxiliary nonlinear block. The loss for structural information preservation backpropagates to the input through an auxiliary lightweight projection that conceptually serves as feedback mediation while the orthogonality constraints account for the boundedness of updating magnitude. Extensive experimental results show that SPHeRe achieves SOTA performance among unsupervised synaptic plasticity approaches on standard image classification benchmarks, including CIFAR-10, CIFAR-100, and Tiny-ImageNet. Furthermore, the method exhibits strong effectiveness in continual learning and transfer learning scenarios, and image reconstruction tasks show the robustness and generalizability of the extracted features. This work demonstrates the competitiveness and potential of Hebbian unsupervised learning rules within modern deep learning frameworks, demonstrating the possibility of efficient and biologically inspired learning algorithms without the strong dependence on strict backpropagation. Our code is available at https://github.com/brain-intelligence-lab/SPHeRe.

Citations

• Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
• Masked Autoencoders Are Scalable Vision Learners
• Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks
• Exploring Simple Siamese Representation Learning
• Exploring Simple Siamese Representation Learning
• Bootstrap your own latent: A new approach to self-supervised Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks
• Local Unsupervised Learning for Image Analysis
• Training Neural Networks with Local Error Signals
• Overcoming catastrophic forgetting with hard attention to the task
• Attention Is All You Need
• Deep Residual Learning for Image Recognition
• Least squares quantization in PCM
• Simplified neuron model as a principal component analyzer
• Analysis of a complex of statistical variables into principal components.

Related

• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks
• Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks
• Learning Sparse, Distributed Representations using the Hebbian Principle
• PyTorch-Hebbian: facilitating local learning in a deep learning framework
• Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks