2020/12/31 by Jiachun Wang, Fajie Yuan, Jian Chen +4
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Advanced Graph Neural Networks #Artificial intelligence #Computer science #Deep learning #Layer (electronics) #Machine learning #Recommender Systems and Techniques #Scratch #Simple (philosophy) #Stacking #cs.IR #cs.LG
paper · pdf · doi:10.1145/3404835.3462890
openalex created_date 2020/12/21 · arxiv created 2021/05/12 · arxiv updated 2021/05/13 · openalex publication_date 2021/07/11 · openalex updated_date 2026/08/06
Deep learning has brought great progress for the sequential recommendation (SR) tasks. With advanced network architectures, sequential recommender models can be stacked with many hidden layers, e.g., up to 100 layers on real-world recommendation datasets. Training such a deep network is difficult because it can be computationally very expensive and takes much longer time, especially in situations where there are tens of billions of user-item interactions. To deal with such a challenge, we present StackRec, a simple, yet very effective and efficient training framework for deep SR models by iterative layer stacking. Specifically, we first offer an important insight that hidden layers/blocks in a well-trained deep SR model have very similar distributions. Enlightened by this, we propose the stacking operation on the pre-trained layers/blocks to transfer knowledge from a shallower model to a deep model, then we perform iterative stacking so as to yield a much deeper but easier-to-train SR model. We validate the performance of StackRec by instantiating it with four state-of-the-art SR models in three practical scenarios with real-world datasets. Extensive experiments show that StackRec achieves not only comparable performance, but also substantial acceleration in training time, compared to SR models that are trained from scratch. Codes are available at https://github.com/wangjiachun0426/StackRec.