RecoGym: A Reinforcement Learning Environment for the problem of Product\n Recommendation in Online Advertising

2018/08/02 by David Rohde, Rohde, David, Stephen Bonner +8 · 15 citations
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Recommender Systems and Techniques #Smart Grid Energy Management

paper · pdf · doi:10.48550/arxiv.1808.00720

openalex publication_date 2018/08/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recommender Systems are becoming ubiquitous in many settings and take many\nforms, from product recommendation in e-commerce stores, to query suggestions\nin search engines, to friend recommendation in social networks. Current\nresearch directions which are largely based upon supervised learning from\nhistorical data appear to be showing diminishing returns with a lot of\npractitioners report a discrepancy between improvements in offline metrics for\nsupervised learning and the online performance of the newly proposed models.\nOne possible reason is that we are using the wrong paradigm: when looking at\nthe long-term cycle of collecting historical performance data, creating a new\nversion of the recommendation model, A/B testing it and then rolling it out. We\nsee that there a lot of commonalities with the reinforcement learning (RL)\nsetup, where the agent observes the environment and acts upon it in order to\nchange its state towards better states (states with higher rewards). To this\nend we introduce RecoGym, an RL environment for recommendation, which is\ndefined by a model of user traffic patterns on e-commerce and the users\nresponse to recommendations on the publisher websites. We believe that this is\nan important step forward for the field of recommendation systems research,\nthat could open up an avenue of collaboration between the recommender systems\nand reinforcement learning communities and lead to better alignment between\noffline and online performance metrics.\n

Citations

• Matrix Factorization Techniques for Recommender Systems
• The central role of the propensity score in observational studies for causal effects

Cited by

• Selecting User Histories to Generate LLM Users for Cold-Start Item Recommendation
• LLM-as-a-Judge: Toward World Models for Slate Recommendation Systems
• Beyond Action Imitation: Learning a Decision-Aware User Simulator for Online Advertising
• Multistakeholder Impacts of Profile Portability in a Recommender Ecosystem
• Simulations in Recommender Systems: An industry perspective
• Three Methods for Training on Bandit Feedback
• Reconsidering Analytical Variational Bounds for Output Layers of Deep Networks
• Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
• Imitate TheWorld: A Search Engine Simulation Platform
• Self-Supervised Reinforcement Learning for Recommender Systems
• Diagnostic-Guided Dynamic Profile Optimization for LLM-based User Simulators in Sequential Recommendation
• Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
• RecSim: A Configurable Simulation Platform for Recommender Systems
• Meta Off-Policy Estimation
• Exploitation Over Exploration: Unmasking the Bias in Linear Bandit Recommender Offline Evaluation

Related

• Optimized Recommender Systems with Deep Reinforcement Learning
• Rethinking Reinforcement Learning for Recommendation: A Prompt Perspective
• DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systems
• Deep Reinforcement Learning-Based Product Recommender for Online Advertising
• RecLLM-R1: A Two-Stage Training Paradigm with Reinforcement Learning and Chain-of-Thought v1