Coded-InvNet for Resilient Prediction Serving Systems

2021/06/11 by Tuan Dinh, Kangwook Lee, Dinh, Tuan +1
Computer Science · Engineering · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Distributed #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2106.06445

openalex publication_date 2021/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Inspired by a new coded computation algorithm for invertible functions, we propose Coded-InvNet a new approach to design resilient prediction serving systems that can gracefully handle stragglers or node failures. Coded-InvNet leverages recent findings in the deep learning literature such as invertible neural networks, Manifold Mixup, and domain translation algorithms, identifying interesting research directions that span across machine learning and systems. Our experimental results show that Coded-InvNet can outperform existing approaches, especially when the compute resource overhead is as low as 10%. For instance, without knowing which of the ten workers is going to fail, our algorithm can design a backup task so that it can correctly recover the missing prediction result with an accuracy of 85.9%, significantly outperforming the previous SOTA by 32.5%.

Citations

• Distilling the Knowledge in a Neural Network
• The tail at scale
• Approximation by superpositions of a sigmoidal function
• Conditional Generative Adversarial Nets
• An Overview of Multi-Task Learning in Deep Neural Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• mixup: Beyond Empirical Risk Minimization
• Manifold Mixup: Better Representations by Interpolating Hidden States
• The Lipschitz Constant of Self-Attention

Related

• NeRCC: Nested-Regression Coded Computing for Resilient Distributed Prediction Serving Systems
• Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation
• ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems
• CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors
• PARIS: Predicting Application Resilience Using Machine Learning