Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking

2025/08/12 by J NI, Ni, Jiani, He Zhao +5
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2508.09116

openalex publication_date 2025/08/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years, deep neural networks (DNNs) have shown competitive results in many fields. Despite this success, they often suffer from poor calibration, especially in safety-critical scenarios such as autonomous driving and healthcare, where unreliable confidence estimates can lead to serious consequences. Recent studies have focused on improving calibration by modifying the classifier, yet such efforts remain limited. Moreover, most existing approaches overlook calibration errors caused by underconfidence, which can be equally detrimental. To address these challenges, we propose MaC-Cal, a novel mask-based classifier calibration method that leverages stochastic sparsity to enhance the alignment between confidence and accuracy. MaC-Cal adopts a two-stage training scheme with adaptive sparsity, dynamically adjusting mask retention rates based on the deviation between confidence and accuracy. Extensive experiments show that MaC-Cal achieves superior calibration performance and robustness under data corruption, offering a practical and effective solution for reliable confidence estimation in DNNs.

Citations

• Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration
• Decoupling Feature Extraction and Classification Layers for Calibrated Neural Networks
• ACLS: Adaptive and Conditional Label Smoothing for Network Calibration
• Dual Focal Loss for Calibration
• Calibrating a Deep Neural Network with Its Predecessors
• The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration
• Masked Autoencoders Are Scalable Vision Learners
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Calibration of Model Uncertainty for Dropout Variational Inference
• Calibrating Deep Neural Networks using Focal Loss
• A Simple Framework for Contrastive Learning of Visual Representations
• Confidence Calibration for Convolutional Neural Networks Using Structured Dropout
• Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• When Does Label Smoothing Help?
• On Mixup Training: Improved Calibration and Predictive Uncertainty for\n Deep Neural Networks
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• DropBlock: A regularization method for convolutional networks
• mixup: Beyond Empirical Risk Minimization
• On Calibration of Modern Neural Networks
• Attention Is All You Need
• Concrete Dropout
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Densely Connected Convolutional Networks
• Wide Residual Networks
• Deep Networks with Stochastic Depth
• Recurrent Dropout Without Memory Loss
• Deep Residual Learning for Image Recognition
• Rethinking the Inception Architecture for Computer Vision
• Posterior calibration and exploratory analysis for natural language processing models
• Efficient Object Localization Using Convolutional Networks
• Improving neural networks by preventing co-adaptation of feature detectors
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration
• Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets
• Multiclass Alignment of Confidence and Certainty for Network Calibration
• Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning
• A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration