Z-Error Loss for Training Neural Networks

2025/06/02 by Guillaume Godin, Godin, Guillaume · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2506.02154

openalex publication_date 2025/06/02 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Outliers introduce significant training challenges in neural networks by propagating erroneous gradients, which can degrade model performance and generalization. We propose the Z-Error Loss, a statistically principled approach that minimizes outlier influence during training by masking the contribution of data points identified as out-of-distribution within each batch. This method leverages batch-level statistics to automatically detect and exclude anomalous samples, allowing the model to focus its learning on the true underlying data structure. Our approach is robust, adaptive to data quality, and provides valuable diagnostics for data curation and cleaning.

Cited by

• Fast Leave-One-Out Approximation from Fragment-Target Prevalence Vectors (molFTP) : From Dummy Masking to Key-LOO for Leakage-Free Feature Construction

Related

• Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown
• Understanding and Minimising Outlier Features in Neural Network Training
• Out-of-Distribution Detection Using Outlier Detection Methods
• Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization
• Outlier Detection through Null Space Analysis of Neural Networks