Monolith: Real Time Recommendation System With Collisionless Embedding Table

2022/09/16 by Zhuoran Liu, Liu, Zhuoran, Leqi Zou +19 · 16 voices · 12 citations
Computer Science · Decision Sciences · #Recommender Systems and Techniques #Advanced Bandit Algorithms Research #Data Stream Mining Techniques

paper · pdf · doi:10.48550/arxiv.2209.07663

Abstract

Building a scalable and real-time recommendation system is vital for many businesses driven by time-sensitive customer feedback, such as short-videos ranking or online ads. Despite the ubiquitous adoption of production-scale deep learning frameworks like TensorFlow or PyTorch, these general-purpose frameworks fall short of business demands in recommendation scenarios for various reasons: on one hand, tweaking systems based on static parameters and dense computations for recommendation with dynamic and sparse features is detrimental to model quality; on the other hand, such frameworks are designed with batch-training stage and serving stage completely separated, preventing the model from interacting with customer feedback in real-time. These issues led us to reexamine traditional approaches and explore radically different design choices. In this paper, we present Monolith, a system tailored for online training. Our design has been driven by observations of our application workloads and production environment that reflects a marked departure from other recommendations systems. Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provide an production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product.

Cited by

• Mutable Low-Rank Sketches for Retrain-Free Recommendation
• Bonsai: Intentional and Personalized Social Media Feeds
• Affect and prediction in short-video social media recommendation algorithm: TikTok and the missing half-second
• Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates
• xGR: Efficient Generative Recommendation Serving at Scale
• Inference Time Feature Injection: A Lightweight Approach for Real-Time Recommendation Freshness
• Efficient Hyperparameter Search for Non-Stationary Model Training
• PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
• Weightless Neural Networks for Continuously Trainable Personalized Recommendation Systems
• Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
• ACM Conference on Recommender Systems [wikipedia]
• Cuckoo hashing [wikipedia]

Discussions

• Monolith: Real time recommendation system with collisionless embedding table [hn, 70 points, 6 comments]
• In case you are curious how TikTok’s algorithm works, here are the blueprints for Monolith. Literally anyone can implement it if they’re competent. No need to pay $100 billion for it. arxiv.org/pdf/22 [bsky, 8 points, 1 comments]
• Monolith: Real Time Recommendation System With Collisionless Embedding Table [lobsters, 3 points, 1 comments]
• Paper on 'Monolith: Real Time Recommendation System' by Bytedance Inc (TikTok) [hn, 3 points, 0 comments]
• TikTok published an article on their recommendation system architecture [hn, 3 points, 0 comments]
• For TikTok, they released part of their realtime recommender system in this paper: arxiv.org/abs/2209.07663 [bsky, 3 points, 1 comments]
• Monolith – The research paper behind TikToks algorithm (2022) [hn, 2 points, 0 comments]
• here's their paper (one of their papers?) about Monolith, which is what drives their For You page i already miss it dearly [bsky, 2 points, 0 comments]
• Real-Time Recommendation System with Collisionless Embedding Table (2022) [pdf] [hn, 2 points, 0 comments]
• Monolith: Real Time Recommendation System with Collisionless Embedding Table [hn, 2 points, 0 comments]
• 2022 Recommendation System paper from ByteDance – the parent company of TikTo [hn, 1 points, 1 comments]
• online trainingのための構成か。TikTok規模だとここまでやるんだな。Parameter Serverの管理とか激重じゃん。hashing trickを現代で見るとは arxiv.org/abs/2209.07663 [bsky, 1 points, 1 comments]
• Monolith: Real Time Recommendation System with Collisionless Embedding Table [hn, 1 points, 0 comments]
• Real Time Recommendation System with Collisionless Embedding Table (2022) [hn, 1 points, 0 comments]
• arxiv.org/pdf/2209.07663 [bsky, 0 points, 0 comments]
• As a parent, I have been very cautious about TikTok’s addictive capabilities. As a computer scientist, I have been at awe. Read this if you’re interested in recommendation systems: arxiv.org/abs/2209. [bsky, 0 points, 1 comments]

Related

• KEEP: An Industrial Pre-Training Framework for Online Recommendation via Knowledge Extraction and Plugging
• Deep Learning Model Acceleration and Optimization Strategies for Real-Time Recommendation Systems
• Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings
• BagPipe: Accelerating Deep Recommendation Model Training
• M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems