Recommender Systems with Generative Retrieval

2023/05/08 by Shashank Rajput, Nikhil Mehta, Rajput, Shashank +25 · 1 voice · 64 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Recommender Systems and Techniques #Topic Modeling #cs.IR #cs.LG

paper · pdf · doi:10.48550/arxiv.2305.05065

openalex publication_date 2023/05/08 · arxiv published 2023/05/08 · arxiv updated 2023/11/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Modern recommender systems perform large-scale retrieval by first embedding queries and item candidates in the same unified space, followed by approximate nearest neighbor search to select top candidates given a query embedding. In this paper, we propose a novel generative retrieval approach, where the retrieval model autoregressively decodes the identifiers of the target candidates. To that end, we create semantically meaningful tuple of codewords to serve as a Semantic ID for each item. Given Semantic IDs for items in a user session, a Transformer-based sequence-to-sequence model is trained to predict the Semantic ID of the next item that the user will interact with. To the best of our knowledge, this is the first Semantic ID-based generative model for recommendation tasks. We show that recommender systems trained with the proposed paradigm significantly outperform the current SOTA models on various datasets. In addition, we show that incorporating Semantic IDs into the sequence-to-sequence model enhances its ability to generalize, as evidenced by the improved retrieval performance observed for items with no prior interaction history.

Cited by

• Tokenizing Numerical and Embedding Features for LLM RecSys
• The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers
• CEMG: Collaborative-Enhanced Multimodal Generative Recommendation
• Efficient Optimization of Hierarchical Identifiers for Generative Recommendation
• On Recommending Category: A Cascading Approach
• From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models
• SPAR: Session-based Pipeline for Adaptive Retrieval on Legacy File Systems
• Breaking the Curse of Dimensionality: On the Stability of Modern Vector Retrieval
• xGR: Efficient Generative Recommendation Serving at Scale
• Foresight Prediction Enhanced Live-Streaming Recommendation
• Selecting User Histories to Generate LLM Users for Cold-Start Item Recommendation
• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• Masked Diffusion for Generative Recommendation
• CoFiRec: Coarse-to-Fine Tokenization for Generative Recommendation
• NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations
• CAT-ID²: Category-Tree Integrated Document Identifier Learning for Generative Retrieval In E-commerce
• A Survey on Generative Recommendation: Data, Model, and Tasks
• WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models
• Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
• MuseTok: Symbolic Music Tokenization for Generation and Semantic Understanding
• MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation
• Pctx: Tokenizing Personalized Context for Generative Recommendation
• Bi-Level Optimization for Generative Recommendation: Bridging Tokenization and Generation
• Towards Context-aware Reasoning-enhanced Generative Searching in E-commerce
• xLLM Technical Report
• CTRL-Rec: Controlling Recommender Systems With Natural Language
• Reinforced Preference Optimization for Recommendation
• Next Interest Flow: A Generative Pre-training Paradigm for Recommender Systems by Modeling All-domain Movelines
• ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval
• PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling
• MATT-CTR: Unleashing a Model-Agnostic Test-Time Paradigm for CTR Prediction with Confidence-Guided Inference Paths
• OneVision: An End-to-End Generative Framework for Multi-view E-commerce Vision Search
• TalkPlay-Tools: Conversational Music Recommendation with LLM Tool Calling
• Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation
• Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation
• Understanding Generative Recommendation with Semantic IDs from a Model-scaling View
• FreeRet: MLLMs as Training-Free Retrievers
• From Past To Path: Masked History Learning for Next-Item Prediction in Generative Recommendation
• Does Generative Retrieval Overcome the Limitations of Dense Retrieval?
• IntSR: An Integrated Generative Framework for Search and Recommendation
• FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasets
• LoopMemGR: From Behavior Logs to Evolving Memory for Generative Recommendation
• Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation
• OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval
• LASAR: Latent Adaptive Semantic Aligned Reasoning for Generative Recommendation
• Aligning Recommendations with User Popularity Preferences
• Equip Pre-ranking with Target Attention by Residual Quantization
• Sequential Data Augmentation for Generative Recommendation
• RecoWorld: Building Simulated Environments for Agentic Recommender Systems
• Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation
• OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search
• Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs
• MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation
• OneRec-V2 Technical Report
• A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directions
• Test-Time Scaling Strategies for Generative Retrieval in Multimodal Conversational Recommendations
• REG4Rec: Reasoning-Enhanced Generative Model for Large-Scale Recommendation Systems
• OneLoc: Geo-Aware Generative Recommender Systems for Local Life Service
• TBGRecall: A Generative Retrieval Model for E-commerce Recommendation Scenarios
• DAS: Dual-Aligned Semantic IDs Empowered Industrial Recommender System
• Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling
• KuaiLive: A Real-time Interactive Dataset for Live Streaming Recommendation
• Benefit from Rich: Tackling Search Interaction Sparsity in Search Enhanced Recommendation
• Generative Bid Shading in Real-Time Bidding Advertising

Discussions

• Generative retrievalという推薦システムの方向性が模索されてるの知らなかった。近年の主流はユーザーやアイテムをベクトル化して近傍探索で推薦するというものだけど、generateive retrievalでは各アイテムをそれぞれに意味のあるトークン (Semantic ID) に分解して、encoder-decoder モデルで自然言語の生成のように推薦アイテムを生成する。Sema [bsky, 3 points, 0 comments]

Related

• Generative Retrieval with Semantic Tree-Structured Item Identifiers via Contrastive Learning
• Unifying Generative and Dense Retrieval for Sequential Recommendation
• EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration
• Semantic IDs for Joint Generative Search and Recommendation
• Bridging Search and Recommendation in Generative Retrieval: Does One Task Help the Other?