2023/06/01 by David Rügamer, Rügamer, David
Computer Science · Engineering · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.2306.00522
openalex publication_date 2023/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent advances to combine structured regression models and deep neural networks for better interpretability, more expressiveness, and statistically valid uncertainty quantification demonstrate the versatility of semi-structured neural networks (SSNs). We show that techniques to properly identify the contributions of the different model components in SSNs, however, lead to suboptimal network estimation, slower convergence, and degenerated or erroneous predictions. In order to solve these problems while preserving favorable model properties, we propose a non-invasive post-hoc orthogonalization (PHO) that guarantees identifiability of model components and provides better estimation and prediction quality. Our theoretical findings are supported by numerical experiments, a benchmark comparison as well as a real-world application to COVID-19 infections.