Failure Prediction Is a Better Performance Proxy for Early-Exit Networks Than Calibration

2025/08/29 by Kubaty, Piotr, Szatkowski, Filip, Jazbec, Metod +1
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2508.21495

Abstract

Early-exit models accelerate inference by attaching internal classifiers to intermediate layers of the network, allowing computation to halt once a prediction meets a predefined exit criterion. Most early-exit methods rely on confidence-based exit strategies, which has motivated prior work to calibrate intermediate classifiers in pursuit of improved performance-efficiency trade-offs. In this paper, we argue that calibration metrics can be misleading indicators of multi-exit model performance. Specifically, we present empirical evidence showing that miscalibrated networks can outperform calibrated ones. As an alternative, we propose using failure prediction as a more informative proxy for early-exit model performance. Unlike calibration, failure prediction captures changes in sample rankings and correlates strongly with efficiency gains, offering a more reliable framework for designing and evaluating early-exit models.

Citations

• FlashThink: An Early Exit Method For Efficient Reasoning
• Dynamic Early Exit in Reasoning Models
• Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
• Early-Exit Neural Networks with Nested Prediction Sets
• Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN
• When Does Confidence-Based Cascade Deferral Suffice?
• Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity
• Fixing Overconfidence in Dynamic Neural Networks
• Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles
• Confident Adaptive Language Modeling
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• FrameExit: Conditional Early Exiting for Efficient Video Recognition
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
• Branchy-GNN: a Device-Edge Co-Inference Framework for Efficient Point Cloud Processing
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• BERT Loses Patience: Fast and Robust Inference with Early Exit
• Consistent Estimators for Learning to Defer to an Expert
• Learning Dynamic Routing for Semantic Segmentation
• Addressing Failure Prediction by Learning Model Confidence
• Verified Uncertainty Calibration
• Improved Techniques for Training Adaptive Deep Networks
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• Evaluating model calibration in classification
• Moment Matching for Multi-Source Domain Adaptation
• mixup: Beyond Empirical Risk Minimization
• On Calibration of Modern Neural Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Deep Residual Learning for Image Recognition
• Conditional Deep Learning for Energy-Efficient and Enhanced Pattern\n Recognition
• Deeply-Supervised Nets

Related

• Understanding the Robustness of Multi-Exit Models under Common Corruptions
• Rethinking Confidence Calibration for Failure Prediction
• Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
• How to Train Your Multi-Exit Model? Analyzing the Impact of Training Strategies
• Learning to Weight Samples for Dynamic Early-exiting Networks