2024/10/05 by Jong-Hyun Park, Juyeop Kim, Park, Jonghyun +3 · 1 citation
Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Fault Detection and Control Systems #Structural Health Monitoring Techniques
paper · pdf · doi:10.48550/arxiv.2410.03999
openalex publication_date 2024/10/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31
Recent studies have shown that regularization techniques using soft labels, e.g., label smoothing, Mixup, and CutMix, not only enhance image classification accuracy but also mitigate miscalibration due to overconfident predictions, and improve robustness against adversarial attacks. However, the underlying mechanisms of such improvements remain underexplored. In this paper, we offer a novel explanation from the perspective of the representation space (i.e., the space of the features obtained at the penultimate layer). Based on examination of decision boundaries and structure of features (or representation vectors), our study investigates confidence contours and gradient directions within the representation space. Furthermore, we analyze the adjustments in feature distributions due to regularization in relation to these contours and directions, from which we uncover central mechanisms inducing improved calibration and robustness. Our findings provide new insights into the characteristics of the high-dimensional representation space in relation to training and regularization using soft labels.