2021/10/28 by Jinpeng Wang, Wang, Jinpeng, Jieming Zhu +3 · 7 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning and ELM #Recommender Systems and Techniques
paper · pdf · doi:10.48550/arxiv.2110.15154
openalex publication_date 2021/10/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The two-tower architecture has been widely applied for learning item and user representations, which is important for large-scale recommender systems. Many two-tower models are trained using various in-batch negative sampling strategies, where the effects of such strategies inherently rely on the size of mini-batches. However, training two-tower models with a large batch size is inefficient, as it demands a large volume of memory for item and user contents and consumes a lot of time for feature encoding. Interestingly, we find that neural encoders can output relatively stable features for the same input after warming up in the training process. Based on such facts, we propose a simple yet effective sampling strategy called Cross-Batch Negative Sampling (CBNS), which takes advantage of the encoded item embeddings from recent mini-batches to boost the model training. Both theoretical analysis and empirical evaluations demonstrate the effectiveness and the efficiency of CBNS.