2020/10/12 by Xiaoyong Yang, Yadong Zhu, Yang, Xiaoyong +7 · 17 citations
Computer Science · Mathematics · #Caching and Content Delivery #Cluster analysis #Computer science #Data mining #Graph #Information retrieval #Machine learning #Mathematics #Parallel computing #Product (mathematics) #Purchasing #Recommender Systems and Techniques #Recommender system #Search engine indexing #Speedup #Theoretical computer science #Web Data Mining and Analysis #cs.IR
paper · pdf · doi:10.48550/arxiv.2010.05525
published in arXiv (Cornell University) (Cornell University)
arxiv created 2020/10/12 · openalex publication_date 2020/10/12 · arxiv updated 2020/10/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Building a recommendation system that serves billions of users on daily basis is a challenging problem, as the system needs to make astronomical number of predictions per second based on real-time user behaviors with O(1) time complexity. Such kind of large scale recommendation systems usually rely heavily on pre-built index of products to speedup the recommendation service so that online user waiting time is un-noticeable. One important indexing structure is the product-product index, where one can retrieval a list of ranked products given a seed product. The index can be viewed as a weighted product-product graph. In this paper, we present our novel technologies to efficiently build such kind of indexed product graphs. In particular, we propose the Swing algorithm to capture the substitute relationships between products, which can utilize the substructures of user-item click bi-partitive graph. Then we propose the Surprise algorithm for the modeling of complementary product relationships, which utilizes product category information and solves the sparsity problem of user co-purchasing graph via clustering technique. Base on these two approaches, we can build the basis product graph for recommendation in Taobao. The approaches are evaluated comprehensively with both offline and online experiments, and the results demonstrate the effectiveness and efficiency of the work.