2025/07/04 by Andrii Dzhoha, Dzhoha, Andrii, Alisa Mironenko +9 · 1 citation
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Machine Learning (cs.LG) #Recommender Systems and Techniques
paper · pdf · doi:10.48550/arxiv.2507.03789
openalex publication_date 2025/07/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Modern sequential recommender systems commonly use transformer-based models for next-item prediction. While these models demonstrate a strong balance between efficiency and quality, integrating interleaving features - such as the query context (e.g., browse category) under which next-item interactions occur - poses challenges. Effectively capturing query context is crucial for refining ranking relevance and enhancing user engagement, as it provides valuable signals about user intent within a session. Unlike item features, historical query context is typically not aligned with item sequences and may be unavailable at inference due to privacy constraints or feature store limitations - making its integration into transformers both challenging and error-prone. This paper analyzes different strategies for incorporating query context into transformers trained with a causal language modeling procedure as a case study. We propose a new method that effectively fuses the item sequence with query context within the attention mechanism. Through extensive offline and online experiments on a large-scale online platform and open datasets, we present evidence that our proposed method is an effective approach for integrating query context to improve model ranking quality in terms of relevance and diversity.