Towards Sharing Task Environments to Support Reproducible Evaluations of Interactive Recommender Systems

2019/09/13 by Andrea Barraza‐Urbina, Barraza-Urbina, Andrea, Mathieu d’Aquin +1
Computer Science · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Data Stream Mining Techniques #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Recommender Systems and Techniques

paper · pdf · doi:10.48550/arxiv.1909.06133

openalex publication_date 2019/09/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Beyond sharing datasets or simulations, we believe the Recommender Systems (RS) community should share Task Environments. In this work, we propose a high-level logical architecture that will help to reason about the core components of a RS Task Environment, identify the differences between Environments, datasets and simulations; and most importantly, understand what needs to be shared about Environments to achieve reproducible experiments. The work presents itself as valuable initial groundwork, open to discussion and extensions.

Related

• RecSim: A Configurable Simulation Platform for Recommender Systems
• Towards Creating a Standardized Collection of Simple and Targeted Experiments to Analyze Core Aspects of the Recommender Systems Problem
• KuaiSim: A Comprehensive Simulator for Recommender Systems
• Simulations in Recommender Systems: An industry perspective
• Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendation