The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems

2025/05/16 by Petr Kasalický, Kasalický, Petr, Martin Spišák +9 · 3 citations
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Machine Learning (cs.LG) #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.2505.11388

openalex publication_date 2025/05/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Industry-scale recommender systems face a core challenge: representing entities with high cardinality, such as users or items, using dense embeddings that must be accessible during both training and inference. However, as embedding sizes grow, memory constraints make storage and access increasingly difficult. We describe a lightweight, learnable embedding compression technique that projects dense embeddings into a high-dimensional, sparsely activated space. Designed for retrieval tasks, our method reduces memory requirements while preserving retrieval performance, enabling scalable deployment under strict resource constraints. Our results demonstrate that leveraging sparsity is a promising approach for improving the efficiency of large-scale recommenders. We release our code at https://github.com/recombee/CompresSAE.

Citations

• Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation
• Discovery architecture for cultural heritage: layered retrieval, institutional authority, and the limits of keyword search
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Post-Training 4-bit Quantization on Embedding Tables
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Simple and Effective Dimensionality Reduction for Word Embeddings
• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs
• Adam: A Method for Stochastic Optimization
• k-Sparse Autoencoders

Cited by

• Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems
• Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Control
• Learning Retrieval Models with Sparse Autoencoders

Related

• Learnable Embedding Sizes for Recommender Systems
• SHARK: A Lightweight Model Compression Approach for Large-scale Recommender Systems
• Scalable Dynamic Embedding Size Search for Streaming Recommendation
• Mixed-Precision Embeddings for Large-Scale Recommendation Models
• Review of compressed embedding layers and their applications for recommender systems