2023/10/06 by Zeyu Yun, Juexiao Zhang, Yun, Zeyu +5
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Artificial intelligence #Artificial neural network #Benchmark (surveying) #Cell Image Analysis Techniques #Cluster analysis #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Digital Imaging for Blood Diseases #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Feature learning #Field (mathematics) #Machine Learning (cs.LG) #Machine learning #Mathematics #Modalities #Pattern recognition (psychology) #Representation (politics) #Self-organizing map #Topology (electrical circuits) #Unsupervised learning
paper · pdf · doi:10.48550/arxiv.2310.04496
openalex publication_date 2023/10/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Unsupervised representation learning has seen tremendous progress. However, it is constrained by its reliance on domain specific stationarity and topology, a limitation not found in biological intelligence systems. For instance, unlike computer vision, human vision can process visual signals sampled from highly irregular and non-stationary sensors. We introduce a novel framework that learns from high-dimensional data without prior knowledge of stationarity and topology. Our model, abbreviated as URLOST, combines a learnable self-organizing layer, spectral clustering, and a masked autoencoder (MAE). We evaluate its effectiveness on three diverse data modalities including simulated biological vision data, neural recordings from the primary visual cortex, and gene expressions. Compared to state-of-the-art unsupervised learning methods like SimCLR and MAE, our model excels at learning meaningful representations across diverse modalities without knowing their stationarity or topology. It also outperforms other methods that are not dependent on these factors, setting a new benchmark in the field. We position this work as a step toward unsupervised learning methods capable of generalizing across diverse high-dimensional data modalities.