Are we really making much progress? A worrying analysis of recent neural recommendation approaches

2019/07/16 by Maurizio Ferrari Dacrema, Paolo Cremonesi, Dietmar Jannach · 9 voices · 26 citations
Computer Science · Decision Sciences · #Recommender Systems and Techniques #Advanced Bandit Algorithms Research #Image Retrieval and Classification Techniques

paper · pdf · doi:10.1145/3298689.3347058

Abstract

Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models.

Cited by

• StARS: Socially Appropriate Robot Actions via a Recommender System-Driven Approach
• Use as Directed? A Comparison of Software Tools Intended to Check Rigor and Transparency of Published Work
• Identifying obfuscated code through graph-based semantic analysis of binary code
• A Position Paper on Recommender Systems in the Era of Autonomous Agents
• A Reproducible and Fair Evaluation of Partition-aware Collaborative Filtering
• PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoders
• Do LLM-judges Align with Human Relevance in Cranfield-style Recommender Evaluation?
• The Value of Personalized Recommendations: Evidence from Netflix
• Have We Really Understood Collaborative Information? An Empirical Investigation
• No-Human in the Loop: Agentic Evaluation at Scale for Recommendation
• Reproducibility in Recommender Systems: A Survey
• Does Weighting Improve Matrix Factorization for Recommender Systems?
• How public datasets constrain the development of diversity-aware news recommender systems, and what law could do about it
• Informed Dataset Selection
• CS-KG 2.0: A Large-scale Knowledge Graph of Computer Science
• Our Model Achieves Excellent Performance on MovieLens: What Does It Mean?
• What Data is Really Necessary? A Feasibility Study of Inference Data Minimization for Recommender Systems
• The Hidden Cost of Defaults in Recommender System Evaluation
• APS Explorer: Navigating Algorithm Performance Spaces for Informed Dataset Selection
• D-RDW: Diversity-Driven Random Walks for News Recommender Systems
• From Entity Reliability to Clean Feedback: An Entity-Aware Denoising Framework Beyond Interaction-Level Signals
• Recommendation Is a Dish Better Served Warm
• Are Multimodal Embeddings Truly Beneficial for Recommendation? A Deep Dive into Whole vs. Individual Modalities
• Maximum Impact with Fewer Features: Efficient Feature Selection for Cold-Start Recommenders through Collaborative Importance Weighting
• Does Multimodality Improve Recommender Systems as Expected? A Critical Analysis and Future Directions
• Discrete-event Tensor Factorization: Learning a Smooth Embedding for Continuous Domains

Discussions

• A Worrying Analysis of Recent Neural Recommendation Approaches [hn, 245 points, 82 comments]
• Are We Making Much Progress? Analysis of Recent Neural Recommendation Approaches [hn, 86 points, 17 comments]
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches [lobsters, 10 points, 2 comments]
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches [lobsters, 5 points, 1 comments]
• Are We Making Much Progress with Neural Networks? [hn, 2 points, 0 comments]
• A Worrying Analysis of Recent Neural Recommendation Approaches [hn, 2 points, 0 comments]
• A Worrying Analysis of Recent Neural Recommendation Approaches [hn, 2 points, 0 comments]
• A Worrying Analysis of Recent Neural Recommendation Approaches [hn, 1 points, 0 comments]
• i liked this paper arxiv.org/abs/1907.06902 gotta love "our field sucks" survey papers [bsky, 1 points, 0 comments]

Related

• A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research
• When Newer is Not Better: Does Deep Learning Really Benefit Recommendation From Implicit Feedback?
• Adaptive Neural Architectures for Recommender Systems
• Predictive accuracy of recommender algorithms
• Neural Hybrid Recommender: Recommendation needs collaboration