RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference

2021/01/29 by Mark Wilkening, Wilkening, Mark, Udit Gupta +11 · 5 citations
Computer Science · #Advanced Data Storage Technologies #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2102.00075

openalex publication_date 2021/01/29 · openalex created_date 2021/02/15 · openalex updated_date 2026/07/28

Abstract

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities. Unfortunately, large and fast DRAM-based memories levy high infrastructure costs. Conventional SSD-based storage solutions offer an order of magnitude larger capacity, but have worse read latency and bandwidth, degrading inference performance. RecSSD is a near data processing based SSD memory system customized for neural recommendation inference that reduces end-to-end model inference latency by 2X compared to using COTS SSDs across eight industry-representative models.

Citations

• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• Neural Collaborative Filtering
• RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
• Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems

Cited by

• Machine Learning-Guided Memory Optimization for DLRM Inference on Tiered Memory
• Cocoon: A System Architecture for Differentially Private Training with Correlated Noises
• ReCross: Efficient Embedding Reduction Scheme for In-Memory Computing using ReRAM-Based Crossbar
• ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Coding
• Request-Only Optimization for Recommendation Systems

Related

• RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
• RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation
• MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions
• SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Models
• DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference