End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

2021/08/10 by Qiwei Chen, Changhua Pei, Chen, Qiwei +9 · 16 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Information Retrieval (cs.IR) #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.2108.04468

openalex publication_date 2021/08/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-k similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and k items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a locality-sensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.

Citations

• Backpropagation Applied to Handwritten Zip Code Recognition
• Session-based Recommendations with Recurrent Neural Networks
• Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
• Multi-Interest Network with Dynamic Routing for Recommendation at Tmall
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction
• Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Cited by

• MUSE: A Simple Yet Effective Multimodal Search-Based Framework for Lifelong User Interest Modeling
• Generative Early Stage Ranking
• DUET: Dual Model Co-Training for Entire Space CTR Prediction
• Decoupled Multimodal Fusion for User Interest Modeling in Click-Through Rate Prediction
• Adaptive User Interest Modeling via Conditioned Denoising Diffusion For Click-Through Rate Prediction
• Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation
• Equip Pre-ranking with Target Attention by Residual Quantization
• VQL: An End-to-End Context-Aware Vector Quantization Attention for Ultra-Long User Behavior Modeling
• Practice on Long Behavior Sequence Modeling in Tencent Advertising
• MARS: Modality-Aligned Retrieval for Sequence Augmented CTR Prediction
• Modeling Long-term User Behaviors with Diffusion-driven Multi-interest Network for CTR Prediction
• Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction
• Exploring Scaling Laws of CTR Model for Online Performance Improvement
• MISS: Multi-Modal Tree Indexing and Searching with Lifelong Sequential Behavior for Retrieval Recommendation
• You Only Evaluate Once: A Tree-based Rerank Method at Meituan
• Aggregate and Broadcast: Scalable and Efficient Feature Interaction for Recommender Systems

Related

• Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction
• Adversarial Filtering Modeling on Long-term User Behavior Sequences for Click-Through Rate Prediction
• Deep Multiple Quantization Network on Long Behavior Sequence for Click-Through Rate Prediction
• A Non-sequential Approach to Deep User Interest Model for CTR Prediction
• Deep Group Interest Modeling of Full Lifelong User Behaviors for CTR Prediction