2025/11/18 by Chen, Zonghao, Nitanda, Atsushi, Gretton, Arthur +1
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Stochastic Gradient Optimization Techniques
paper · doi:10.48550/arxiv.2511.14710
openalex publication_date 2025/11/18 · openalex created_date 2025/11/20 · openalex updated_date 2026/07/28
We establish the first global convergence result of neural networks for two stage least squares (2SLS) approach in nonparametric instrumental variable regression (NPIV). This is achieved by adopting a lifted perspective through mean-field Langevin dynamics (MFLD), unlike standard MFLD, however, our setting of 2SLS entails a bilevel optimization problem in the space of probability measures. To address this challenge, we leverage the penalty gradient approach recently developed for bilevel optimization which formulates bilevel optimization as a Lagrangian problem. This leads to a novel fully first-order algorithm, termed F2BMLD. Apart from the convergence bound, we further provide a generalization bound, revealing an inherent trade-off in the choice of the Lagrange multiplier between optimization and statistical guarantees. Finally, we empirically validate the effectiveness of the proposed method on an offline reinforcement learning benchmark.