2023/01/13 by Loris Felardos, Felardos, Loris, Jérôme Hénin +3 · 2 citations
Engineering · Materials Science · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Physical sciences #Lattice Boltzmann Simulation Studies #Machine Learning (cs.LG) #Machine Learning in Materials Science #Model Reduction and Neural Networks #Statistical Mechanics (cond-mat.stat-mech)
paper · pdf · doi:10.48550/arxiv.2301.05475
openalex publication_date 2023/01/13 · openalex created_date 2023/01/14 · openalex updated_date 2026/07/28
Generating a Boltzmann distribution in high dimension has recently been achieved with Normalizing Flows, which enable fast and exact computation of the generated density, and thus unbiased estimation of expectations. However, current implementations rely on accurate training data, which typically comes from computationally expensive simulations. There is therefore a clear incentive to train models with incomplete or no data by relying solely on the target density, which can be obtained from a physical energy model (up to a constant factor). For that purpose, we analyze the properties of standard losses based on Kullback-Leibler divergences. We showcase their limitations, in particular a strong propensity for mode collapse during optimization on high-dimensional distributions. We then propose strategies to alleviate these issues, most importantly a new loss function well-grounded in theory and with suitable optimization properties. Using as a benchmark the generation of 3D molecular configurations, we show on several tasks that, for the first time, imperfect pre-trained models can be further optimized in the absence of training data.