Towards Amortized Ranking-Critical Training for Collaborative Filtering

2019/06/10 by Sam Lobel, Chunyuan Li, Lobel, Sam +5 · 3 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #Artificial intelligence #Artificial neural network #Code (set theory) #Collaborative filtering #Computer science #Data mining #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Ground truth #Information Retrieval (cs.IR) #Learning to rank #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Process (computing) #Quality (philosophy) #Rank (graph theory) #Ranking (information retrieval) #Recommender Systems and Techniques #Recommender system #Reinforcement learning #cs.IR #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1906.04281

published in arXiv (Cornell University) (Cornell University) · The first two authors contributed equally to this manuscript. Code: https://github.com/samlobel/RaCT_CF

openalex publication_date 2019/06/10 · arxiv created 2020/02/10 · arxiv updated 2020/02/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Collaborative filtering is widely used in modern recommender systems. Recent research shows that variational autoencoders (VAEs) yield state-of-the-art performance by integrating flexible representations from deep neural networks into latent variable models, mitigating limitations of traditional linear factor models. VAEs are typically trained by maximizing the likelihood (MLE) of users interacting with ground-truth items. While simple and often effective, MLE-based training does not directly maximize the recommendation-quality metrics one typically cares about, such as top-N ranking. In this paper we investigate new methods for training collaborative filtering models based on actor-critic reinforcement learning, to directly optimize the non-differentiable quality metrics of interest. Specifically, we train a critic network to approximate ranking-based metrics, and then update the actor network (represented here by a VAE) to directly optimize against the learned metrics. In contrast to traditional learning-to-rank methods that require to re-run the optimization procedure for new lists, our critic-based method amortizes the scoring process with a neural network, and can directly provide the (approximate) ranking scores for new lists. Empirically, we show that the proposed methods outperform several state-of-the-art baselines, including recently-proposed deep learning approaches, on three large-scale real-world datasets. The code to reproduce the experimental results and figure plots is on Github: https://github.com/samlobel/RaCT_CF

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Auto-Encoding Variational Bayes
• Cumulated gain-based evaluation of IR techniques
• Conditional Generative Adversarial Nets
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Sequence Level Training with Recurrent Neural Networks
• Deep Reinforcement Learning for Dialogue Generation
• An Actor-Critic Algorithm for Sequence Prediction
• Connecting Generative Adversarial Networks and Actor-Critic Methods
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Adversarial Ranking for Language Generation
• cGANs with Projection Discriminator
• Outer Product-based Neural Collaborative Filtering
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Adversarial Learning for Neural Dialogue Generation
• Deep Reinforcement Learning-based Image Captioning with Embedding Reward
• Variational Inference: A Review for Statisticians

Cited by

• On the Mechanisms of Collaborative Learning in VAE Recommenders

Related

• Training Deep AutoEncoders for Collaborative Filtering
• Deep Variational Models for Collaborative Filtering-based Recommender Systems
• Neural Collaborative Autoencoder
• Adversarial Collaborative Auto-encoder for Top-N Recommendation
• Variational Autoencoders for Collaborative Filtering