2022/07/17 by Lorenz Vaitl, Kim A. Nicoli, Vaitl, Lorenz +5 · 2 citations
Computer Science · Physics and Astronomy · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.2207.08219
openalex publication_date 2022/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We propose an algorithm to estimate the path-gradient of both the reverse and forward Kullback-Leibler divergence for an arbitrary manifestly invertible normalizing flow. The resulting path-gradient estimators are straightforward to implement, have lower variance, and lead not only to faster convergence of training but also to better overall approximation results compared to standard total gradient estimators. We also demonstrate that path-gradient training is less susceptible to mode-collapse. In light of our results, we expect that path-gradient estimators will become the new standard method to train normalizing flows for variational inference.