Conformal Prediction for Deep Classifier via Label Ranking

2023/10/10 by Jianguo Huang, H. Xi, Huang, Jianguo +9 · 11 citations
Computer Science · #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2310.06430

openalex publication_date 2023/10/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Conformal prediction is a statistical framework that generates prediction sets containing ground-truth labels with a desired coverage guarantee. The predicted probabilities produced by machine learning models are generally miscalibrated, leading to large prediction sets in conformal prediction. To address this issue, we propose a novel algorithm named Sorted Adaptive Prediction Sets (SAPS), which discards all the probability values except for the maximum softmax probability. The key idea behind SAPS is to minimize the dependence of the non-conformity score on the probability values while retaining the uncertainty information. In this manner, SAPS can produce compact prediction sets and communicate instance-wise uncertainty. Extensive experiments validate that SAPS not only lessens the prediction sets but also broadly enhances the conditional coverage rate of prediction sets.

Cited by

• Conformal Correction for Efficiency May be at Odds with Entropy
• CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression
• Enhancing Conformal Prediction via Class Similarity
• Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds
• Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification
• Quantifying and Improving Adaptivity in Conformal Prediction through Input Transformations
• Document Summarization with Conformal Importance Guarantees
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
• Performance of Conformal Prediction in Capturing Aleatoric Uncertainty
• Online Conformal Selection with Accept-to-Reject Changes
• Conformal Prediction Meets Long-tail Classification

Related

• C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Sets
• Trustworthy Classification through Rank-Based Conformal Prediction Sets
• Direct Prediction Set Minimization via Bilevel Conformal Classifier Training
• Improving Uncertainty Quantification of Deep Classifiers via Neighborhood Conformal Prediction: Novel Algorithm and Theoretical Analysis
• Sparse Activations as Conformal Predictors