Diversity regularization in deep ensembles

2018/02/22 by Changjian Shui, Shui, Changjian, Azadeh Sadat Mozafari +7 · 2 citations
Computer Science · Mathematics · #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Neural Networks and Applications #Statistical Methods and Inference #cs.LG

paper · pdf · doi:10.48550/arxiv.1802.07881

6 pages

arxiv created 2018/02/22 · openalex publication_date 2018/02/22 · arxiv updated 2018/02/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Calibrating the confidence of supervised learning models is important for a variety of contexts where the certainty over predictions should be reliable. However, it has been reported that deep neural network models are often too poorly calibrated for achieving complex tasks requiring reliable uncertainty estimates in their prediction. In this work, we are proposing a strategy for training deep ensembles with a diversity function regularization, which improves the calibration property while maintaining a similar prediction accuracy.

Citations

• End to End Learning for Self-Driving Cars
• On Calibration of Modern Neural Networks
• On Fairness and Calibration

Cited by

• Exploring the Rashomon Set for Concept-Based Models
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization

Related

• Diverse Ensembles Improve Calibration
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• Regularized Neural Ensemblers
• Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles
• On Joint Regularization and Calibration in Deep Ensembles