The tail at scale

2013/01/29 by Jeffrey Dean, Jay B. Dean, Luiz André Barroso · 80 citations
Computer Science · #Distributed systems and fault tolerance #Cloud Computing and Resource Management #Peer-to-Peer Network Technologies

paper · doi:10.1145/2408776.2408794

Abstract

Software techniques that tolerate latency variability are vital to building responsive large-scale Web services.

Cited by

• DDSketch
• Pipelined Gradient Coding
• Hardware-Transparent I/O Governance in Disaggregated Heterogeneous Storage
• OptiNIC: A Resilient and Tail-Optimal RDMA NIC for Distributed ML Workloads
• Robust Gradient Descent via Moment Encoding with LDPC Codes
• Application-Driven Architecture Exploration for Cross-Layer Heterogeneous Systems
• CAPS: Fine-Tuning CCA Timing
• Distributed Optimization using Heterogeneous Compute Systems
• BALLAST: Bandit-Assisted Learning for Latency-Aware Stable Timeouts in Raft
• HADFL: Heterogeneity-aware Decentralized Federated Learning Framework
• Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers
• Lotus: Optimizing Disaggregated Transactions with Disaggregated Locks
• Performance and Stability of Barrier Mode Parallel Systems with Heterogeneous and Redundant Jobs
• FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices
• Efficient Replication for Straggler Mitigation in Distributed Computing
• Optimal Load Balancing in Bipartite Graphs
• Train While You Fight -- Technical Requirements for Advanced Distributed Learning Platforms
• Formal Specification for Fast ACS: Low-Latency File-Based Ordered Message Delivery at Scale
• YTrace: End-to-end Performance Diagnosis in Large Cloud and Content Providers
• Ksurf-Drone: Attention Kalman Filter for Contextual Bandit Optimization in Cloud Resource Allocation
• Coordination-Free Lane Partitioning for Convergent ANN Search
• Understanding Training Efficiency of Deep Learning Recommendation Models at Scale
• Tail-Tolerant Distributed Search
• Contrasting Effects of Replication in Parallel Systems: From Overload to Underload and Back
• Asynchronous Load Balancing and Auto-Scaling: Mean-Field Limit and Optimal Design
• Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy
• Railgun: managing large streaming windows under MAD requirements
• TridentServe: A Stage-level Serving System for Diffusion Pipelines
• Constraint Satisfaction Approaches to Wordle: Novel Heuristics and Cross-Lexicon Validation
• Effective Straggler Mitigation: Which Clones Should Attack and When?
• Reimagining RDMA Through the Lens of ML
• Recursive Manifold Coherence for Deadtime-Aware Distributed Triggering through Geometric State Estimation
• LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learning
• What Limits Agentic Systems Efficiency?
• MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
• REVISION:Reflective Intent Mining and Online Reasoning Auxiliary for E-commerce Visual Search System Optimization
• HGraphScale: Hierarchical Graph Learning for Autoscaling Microservice Applications in Container-based Cloud Computing
• Processing Data Where It Makes Sense: Enabling In-Memory Computation
• Diversity/Parallelism Trade-off in Distributed Systems with Redundancy
• Polynomially Coded Regression: Optimal Straggler Mitigation via Data Encoding
• Towards Stochastically Optimizing Data Computing Flows
• CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors
• Structure-Preserving Error-Correcting Codes for Polynomial Frames
• The Architectural Implications of Microservices in the Cloud
• Zephyrus: Scaling Gateways Beyond the Petabit-Era with DPU-Augmented Hierarchical Co-Offloading
• When Do Redundant Requests Reduce Latency ?
• Information-theoretically Secure Erasure Codes for Distributed Storage
• Adaptive Execution Scheduler for DataDios SmartDiff
• Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications
• Detecting and Preventing Latent Risk Accumulation in High-Performance Software Systems
• SPARQ: An Optimization Framework for the Distribution of AI-Intensive Applications under Non-Linear Delay Constraints
• Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
• Let’s trace it: Fine-grained serverless benchmarking for synchronous and asynchronous applications
• HiveMind: A Scalable and Serverless Coordination Control Platform for UAV Swarms
• Resilient AI Supercomputer Networking using MRC and SRv6
• To Centralize or Not to Centralize: A Tale of Swarm Coordination
• Tell-Tale Tail Latencies: Pitfalls and Perils in Database Benchmarking
• Piquantε: Private Quantile Estimation in the Two-Server Model
• An Empirical Study on Failures in Automated Issue Solving
• Catalpa: GC for a Low-Variance Software Stack
• A Workload and Programming Ease Driven Perspective of\n Processing-in-Memory
• A Uniqueness Theorem for Distributed Computation under Physical Constraint
• DISTRIBUTEDANN: Efficient Scaling of a Single DISKANN Graph Across Thousands of Computers
• Coded Fourier Transform
• Efficient Straggler Replication in Large-scale Parallel Computing
• Fork and Join Queueing Networks with Heavy Tails: Scaling Dimension and Throughput Limit
• CASPR: Judiciously Using the Cloud for Wide-Area Packet Recovery
• Efficient and Scalable Agentic AI with Heterogeneous Systems
• Optimizing Redundancy Levels in Master-Worker Compute Clusters for Straggler Mitigation
• DDB: Source-Level Interactive Debugging for Distributed Applications
• DiNoDB: an Interactive-speed Query Engine for Ad-hoc Queries on Temporary Data
• Collage Inference: Using Coded Redundancy for Low Variance Distributed Image Classification
• Deconstructing the Tail at Scale Effect Across Network Protocols
• ThrottleBot - Performance without Insight
• Coding for Large-Scale Distributed Machine Learning. [europepmc]
• Network Coding Approaches for Distributed Computation over Lossy Wireless Networks. [europepmc]
• Adaptive Privacy-Preserving Coded Computing with Hierarchical Task Partitioning. [europepmc]
• Cost-Efficient Distributed Learning via Combinatorial Multi-Armed Bandits. [europepmc]
• Beyond mimicry: a framework for evaluating genuine intelligence in artificial systems. [europepmc]
• TailX: Scheduling Heterogeneous Multiget Queries to Improve Tail Latencies in Key-Value Stores [europepmc]

Related

• Evolving system bottlenecks in the as a service cloud
• FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices
• Latency-Sensitive Web Service Workflows: A Case for a Software-Defined\n Internet
• Reducing Tail Latencies Through Environment- and Neighbour-aware Thread Management
• The Promise and Challenge of Runtime Variability