Approximation Algorithms for Cascading Prediction Models

2018/02/21 by M. J. V. Streeter, Streeter, Matthew · 1 citation
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1802.07697

openalex publication_date 2018/02/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present an approximation algorithm that takes a pool of pre-trained models as input and produces from it a cascaded model with similar accuracy but lower average-case cost. Applied to state-of-the-art ImageNet classification models, this yields up to a 2x reduction in floating point multiplications, and up to a 6x reduction in average-case memory I/O. The auto-generated cascades exhibit intuitive properties, such as using lower-resolution input for easier images and requiring higher prediction confidence when using a computationally cheaper model.

Citations

• ImageNet Large Scale Visual Recognition Challenge
• Adaptive Computation Time for Recurrent Neural Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Cited by

• Cascadia: An Efficient Cascade Serving System for Large Language Models

Related

• Structured Prediction Cascades
• BasisNet: Two-stage Model Synthesis for Efficient Inference
• Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems
• IDK Cascades: Fast Deep Learning by Learning not to Overthink
• Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time